Detection of Disease Features on Retinal OCT Scans Using RETFound
Abstract
1. Introduction
2. Methods
2.1. Dataset
2.2. Feature Description
2.3. Annotation Tool
2.4. Annotation Strategy
2.5. Data Preprocessing
2.6. RETFound Model
2.7. ResNet-50 Model
3. Results
4. Discussion
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Fleckenstein, M.; Keenan, T.D.; Guymer, R.H.; Chakravarthy, U.; Schmitz-Valckenberg, S.; Klaver, C.C.; Wong, W.T.; Chew, E.Y. Age-related macular degeneration. Nat. Rev. Dis. Primers 2021, 7, 31. [Google Scholar] [CrossRef] [PubMed]
- Kollias, A.N.; Ulbig, M.W. Diabetic retinopathy: Early diagnosis and effective treatment. Dtsch. Arztebl. Int. 2010, 107, 75. [Google Scholar] [PubMed]
- Aumann, S.; Donner, S.; Fischer, J.; Müller, F. Optical coherence tomography (OCT): Principle and technical realization. In High Resolution Imaging in Microscopy and Ophthalmology: New Frontiers in Biomedical Optics; Springer: Cham, Switzerland, 2019; pp. 59–85. [Google Scholar]
- Zeppieri, M.; Marsili, S.; Enaholo, E.S.; Shuaibu, A.O.; Uwagboe, N.; Salati, C.; Spadea, L.; Musa, M. Optical coherence tomography (OCT): A brief look at the uses and technological evolution of ophthalmology. Medicina 2023, 59, 2114. [Google Scholar] [CrossRef]
- Dahrouj, M.; Miller, J.B. Artificial intelligence (AI) and retinal optical coherence tomography (OCT). Semin. Ophthalmol. 2021, 36, 341–345. [Google Scholar] [CrossRef] [PubMed]
- Elezaby, S.; Bagherinia, H.; Ren, H.; Sha, P.; Tracewell, L.; Wu, C.; Fard, A.; Durbin, M. A machine learning method for optical coherence tomography scan quality assessment. Investig. Ophthalmol. Vis. Sci. 2020, 61, PB0090. [Google Scholar]
- Kho, A.; Bagherinia, H.; Leahy, C.; Bello, S.A. Automated scan quality assessment in low-cost OCT. Investig. Ophthalmol. Vis. Sci. 2022, 63, 3313-F0122. [Google Scholar]
- Andreu-Perez, J.; Poon, C.C.; Merrifield, R.D.; Wong, S.T.; Yang, G.Z. Big data for health. IEEE J. Biomed. Health Inform. 2015, 19, 1193–1208. [Google Scholar] [CrossRef]
- Mallappallil, M.; Sabu, J.; Gruessner, A.; Salifu, M. A review of big data and medical research. SAGE Open Med. 2020, 8, 2050312120934839. [Google Scholar] [CrossRef]
- Lee, C.H.; Yoon, H.J. Medical big data: Promise and challenges. Kidney Res. Clin. Pract. 2017, 36, 3–11. [Google Scholar] [CrossRef] [PubMed]
- Jee, K.; Kim, G.H. Potentiality of big data in the medical sector: Focus on how to reshape the healthcare system. Healthc. Inform. Res. 2013, 19, 79–85. [Google Scholar] [CrossRef]
- Price, W.N.; Cohen, I.G. Privacy in the age of medical big data. Nat. Med. 2019, 25, 37–43. [Google Scholar] [CrossRef] [PubMed]
- Liao, H.; Tang, M.; Luo, L.; Li, C.; Chiclana, F.; Zeng, X.J. A bibliometric analysis and visualization of medical big data research. Sustainability 2018, 10, 166. [Google Scholar] [CrossRef]
- Merelli, I.; Pérez-Sánchez, H.; Gesing, S.; D’Agostino, D. Managing, analysing, and integrating big data in medical bioinformatics: Open problems and future perspectives. BioMed Res. Int. 2014, 2014, 134023. [Google Scholar] [CrossRef] [PubMed]
- Cheng, C.Y.; Da Soh, Z.; Majithia, S.; Thakur, S.; Rim, T.H.; Tham, Y.C.; Wong, T.Y. Big data in ophthalmology. Asia-Pac. J. Ophthalmol. 2020, 9, 291–298. [Google Scholar] [CrossRef] [PubMed]
- Lee, C.S.; Brandt, J.D.; Lee, A.Y. Big data and artificial intelligence in ophthalmology: Where are we now? Ophthalmol. Sci. 2021, 1, 100036. [Google Scholar] [CrossRef]
- Da Soh, Z.; Cheng, C.Y. Application of big data in ophthalmology. Taiwan J. Ophthalmol. 2023, 13, 123–132. [Google Scholar] [CrossRef]
- Clark, A.; Ng, J.Q.; Morlet, N.; Semmens, J.B. Big data and ophthalmic research. Surv. Ophthalmol. 2016, 61, 443–465. [Google Scholar] [CrossRef]
- Devarakonda, S.T.; Vupparaboina, K.K.; Richhariya, A.; Chhablani, J.; Jana, S. Automated detection of retinal disorders from OCT images using artificial neural network. In Proceedings of the 2016 IEEE Annual India Conference (INDICON), Bengaluru, India, 16 December 2016; pp. 1–6. [Google Scholar]
- Singh, L.K.; Garg, H.; Khanna, M. An artificial intelligence-based smart system for early glaucoma recognition using OCT images. In Research Anthology on Improving Medical Imaging Techniques for Analysis and Intervention; IGI Global: Hershey, PA, USA, 2023; pp. 1424–1454. [Google Scholar]
- Qiu, J.; Sun, Y. Self-supervised iterative refinement learning for macular OCT volumetric data classification. Comput. Biol. 2019, 111, 103327. [Google Scholar] [CrossRef]
- ElTanboly, A.; Shalaby, A.; Mahmoud, A.; Ghazal, M.; Switala, A.; Taher, F.; Suri, J.S.; Keynton, R.; El-Baz, A. Computer Aided Diagnosis System for Early Detection of Diabetic Retinopathy Using OCT Images. In Big Data in Multimodal Medical Imaging; CRC Press: Boca Raton, FL, USA, 2019; pp. 281–299. [Google Scholar]
- DeBuc, D.C. A review of algorithms for segmentation of retinal image data using optical coherence tomography. Image Segmentation 2011, 1, 15–54. [Google Scholar]
- Kumar, Y.; Gupta, S. Deep transfer learning approaches to predict glaucoma, cataract, choroidal neovascularization, diabetic macular edema, drusen and healthy eyes: An experimental review. Arch. Comput. Methods Eng. 2022, 30, 521–541. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Hassan, E.; Elmougy, S.; Ibraheem, M.R.; Hossain, M.S.; AlMutib, K.; Ghoneim, A.; AlQahtani, S.A.; Talaat, F.M. Enhanced deep learning model for classification of retinal optical coherence tomography images. Sensors 2023, 23, 5393. [Google Scholar] [CrossRef] [PubMed]
- Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2818–2826. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. arXiv 2014, arXiv:1409.1556. [Google Scholar]
- Subasi, M.E.; Patnaik, S.; Subasi, A. Optical coherence tomography image classification for retinal disease detection using artificial intelligence. In Applications of Artificial Intelligence Healthcare and Biomedicine; Academic Press: Cambridge, MA, USA, 2024; pp. 289–323. [Google Scholar]
- Li, F.; Chen, H.; Liu, Z.; Zhang, X.D.; Jiang, M.S.; Wu, Z.Z.; Zhou, K.Q. Deep learning-based automated detection of retinal diseases using optical coherence tomography images. Biomed. Opt. Express 2019, 10, 6204–6226. [Google Scholar] [CrossRef] [PubMed]
- Leandro, I.; Lorenzo, B.; Aleksandar, M.; Dario, M.; Rosa, G.; Agostino, A.; Daniele, T. Oct-based deep-learning models for the identification of retinal key signs. Sci. Rep. 2023, 13, 1–11. [Google Scholar] [CrossRef] [PubMed]
- Bommasani, R.; Hudson, D.A.; Adeli, E.; Altman, R.; Arora, S.; von Arx, S.; Bernstein, M.S.; Bohg, J.; Bosselut, A.; Brunskill, E.; et al. On the opportunities and risks of foundation models. arXiv 2021, arXiv:2108.07258. [Google Scholar]
- Moor, M.; Banerjee, O.; Abad, Z.S.; Krumholz, H.M.; Leskovec, J.; Topol, E.J.; Rajpurkar, P. Foundation models for generalist medical artificial intelligence. Nature 2023, 616, 259–265. [Google Scholar] [CrossRef]
- Zhou, C.; Li, Q.; Li, C.; Yu, J.; Liu, Y.; Wang, G.; Zhang, K.; Ji, C.; Yan, Q.; He, L.; et al. A comprehensive survey on pretrained foundation models: A history from bert to chatgpt. arXiv 2023, arXiv:2302.09419. [Google Scholar] [CrossRef]
- Zhong, W.; Cui, R.; Guo, Y.; Liang, Y.; Lu, S.; Wang, Y.; Saied, A.; Chen, W.; Duan, N. Agieval: A human-centric benchmark for evaluating foundation models. arXiv 2023, arXiv:2304.06364. [Google Scholar]
- Vorontsov, E.; Bozkurt, A.; Casson, A.; Shaikovski, G.; Zelechowski, M.; Severson, K.; Zimmermann, E.; Hall, J.; Tenenholtz, N.; Fusi, N.; et al. A foundation model for clinical-grade computational pathology and rare cancers detection. Nat. Med. 2024, 30, 2924–2935. [Google Scholar] [CrossRef]
- Hong, D.; Zhang, B.; Li, X.; Li, Y.; Li, C.; Yao, J.; Yokoya, N.; Li, H.; Ghamisi, P.; Jia, X.; et al. SpectralGPT: Spectral remote sensing foundation model. IEEE Trans. Pattern Anal. Mach. Intell. 2024, 46, 5227–5244. [Google Scholar] [CrossRef]
- Liang, Y.; Wen, H.; Nie, Y.; Jiang, Y.; Jin, M.; Song, D.; Pan, S.; Wen, Q. Foundation models for time series analysis: A tutorial and survey. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain, 25–29 August 2024; pp. 6555–6565. [Google Scholar]
- Wu, C.; Yin, S.; Qi, W.; Wang, X.; Tang, Z.; Duan, N. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv 2023, arXiv:2303.04671. [Google Scholar]
- Nguyen, T.; Brandstetter, J.; Kapoor, A.; Gupta, J.K.; Grover, A. ClimaX: A foundation model for weather and climate. arXiv 2023, arXiv:2301.10343. [Google Scholar]
- Wang, J.; Chen, D.; Wu, Z.; Luo, C.; Zhou, L.; Zhao, Y.; Xie, Y.; Liu, C.; Jiang, Y.G.; Yuan, L. Omnivl: One foundation model for image-language and video-language tasks. Adv. Neural Inf. Process. Syst. 2022, 35, 5696–5710. [Google Scholar]
- Zhou, Y.; Chia, M.A.; Wagner, S.K.; Ayhan, M.S.; Williamson, D.J.; Struyven, R.R.; Liu, T.; Xu, M.; Lozano, M.G.; Woodward-Court, P.; et al. A foundation model for generalizable disease detection from retinal images. Nature 2023, 622, 156–163. [Google Scholar] [CrossRef] [PubMed]
- Du, K.; Shah, S.; Gadari, A.; Bollepalli, S.C.; Chhablani, J.; Vupparaboina, K.K. Inter-observer variance in labeling quality and pathology of retinal optical coherence tomography scans [abstract]. In Proceedings of the 2024 Data Science and AI Symposium, Harvard Ophthalmology and Mass Eye & Ear, Boston, MA, USA, 26–27 September 2024; p. 14. [Google Scholar]
- He, K.; Chen, X.; Xie, S.; Li, Y.; Dollár, P.; Girshick, R. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 18–24 June 2022; pp. 16000–16009. [Google Scholar]
- Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; IEEE: Piscataway, NJ, USA, 2009; pp. 248–255. [Google Scholar]
- Kermany, D.S.; Goldbaum, M.; Cai, W.; Valentim, C.C.; Liang, H.; Baxter, S.L.; McKeown, A.; Yang, G.; Wu, X.; Yan, F.; et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 2018, 172, 1122–1131. [Google Scholar] [CrossRef]
- Kugelman, J.; Alonso-Caneiro, D.; Read, S.A.; Vincent, S.J.; Chen, F.K.; Collins, M.J. Effect of altered OCT image quality on deep learning boundary segmentation. IEEE Access 2020, 8, 43537–43553. [Google Scholar] [CrossRef]
- Wang, J.; Deng, G.; Li, W.; Chen, Y.; Gao, F.; Liu, H.; He, Y.; Shi, G. Deep learning for quality assessment of retinal OCT images. Biomed. Opt. Express 2019, 10, 6057–6072. [Google Scholar] [CrossRef]
- Lauermann, J.L.; Treder, M.; Alnawaiseh, M.; Clemens, C.R.; Eter, N.; Alten, F. Automated OCT angiography image quality assessment using a deep learning algorithm. Graefes Arch. Clin. Exp. Ophthalmol. 2019, 257, 1641–1648. [Google Scholar] [CrossRef]
Training Set (1360 Scans) | Testing Set (410 Scans) | |
---|---|---|
Healthy scan | 420 | 165 |
Diseased scan | 940 | 245 |
Foveal scan | 135 | 33 |
Subretinal fluid | 89 | 2 |
Intraretinal fluid | 69 | 6 |
Drusen | 227 | 91 |
Pigment epithelial detachment | 220 | 27 |
Hyperreflective dots | 840 | 257 |
Hyperreflective foci | 40 | 32 |
Evaluation Metrics | ||||||||
---|---|---|---|---|---|---|---|---|
Accuracy | Sensitivity | Specificity | AUC-ROC | |||||
Single Task | RN50 | RF | RN50 | RF | RN50 | RF | RN50 | RF |
H/D | 0.77 | 0.76 | 0.71 | 0.62 | 0.81 | 0.86 | 0.83 | 0.80 |
Foveal scan | 0.94 | 0.94 | 0.88 | 0.94 | 0.95 | 0.93 | 0.94 | 0.95 |
Drusen | 0.69 * | 0.76 * | 0.65 * | 0.78 * | 0.70 | 0.75 | 0.75 | 0.83 |
PED | 0.84 * | 0.79* | 0.44 * | 0.63 * | 0.86 | 0.81 | 0.74 | 0.76 |
H-Dots | 0.62 | 0.62 | 0.82 * | 0.67 * | 0.28 * | 0.54 * | 0.64 | 0.66 |
Average | 0.77 | 0.77 | 0.70 | 0.73 | 0.72 | 0.78 | 0.78 | 0.80 |
Multitask | RN50 | RF | RN50 | RF | RN50 | RF | RN50 | RF |
H/D | 0.78 * | 0.74 * | 0.66 | 0.72 | 0.87 * | 0.76 * | 0.80 | 0.81 |
Foveal scan | 0.93 | 0.94 | 0.94 * | 0.85 * | 0.93 | 0.94 | 0.96 | 0.91 |
Drusen | 0.70 * | 0.60 * | 0.66 * | 0.44 * | 0.72 | 0.65 | 0.74 * | 0.60 * |
PED | 0.82 | 0.81 | 0.37 | 0.48 | 0.84 | 0.84 | 0.69 | 0.74 |
H-Dots | 0.62 | 0.64 | 0.80 | 0.72 | 0.30 * | 0.50 * | 0.65 | 0.67 |
Average | 0.77 | 0.75 | 0.69 | 0.64 | 0.73 | 0.74 | 0.76 | 0.75 |
Evaluation Metrics | ||||||||
---|---|---|---|---|---|---|---|---|
Accuracy | Sensitivity | Specificity | AUC-ROC | |||||
Single Task | RN50 | RF | RN50 | RF | RN50 | RF | RN50 | RF |
H/D | 0.94 | 0.94 | 0.90 * | 0.92 * | 0.95 | 0.95 | 0.98 | 0.99 |
Drusen | 0.94 | 0.94 | 0.91 * | 0.92 * | 0.95 | 0.95 | 0.98 | 0.98 |
Evaluation Metrics | ||||||||
---|---|---|---|---|---|---|---|---|
Accuracy | Sensitivity | Specificity | AUC-ROC | |||||
Internal Dataset | ||||||||
Single task | RN50 | RF | RN50 | RF | RN50 | RF | RN50 | RF |
H/D | 0.69 | 0.57 | 0.61 | 0.58 | 0.74 * | 0.56 * | 0.75 * | 0.59 * |
Foveal scan | 0.74 | 0.70 | 0.82 * | 0.97 * | 0.73 | 0.68 | 0.79 * | 0.91 * |
Drusen | 0.59 | 0.69 | 0.57 | 0.56 | 0.60 * | 0.72 * | 0.56 | 0.66 |
PED | 0.81 * | 0.50 * | 0.26 * | 0.78 * | 0.85 * | 0.48 * | 0.64 | 0.63 |
H-Dots | 0.51 * | 0.68 * | 0.36 * | 0.71 * | 0.75 * | 0.62 * | 0.54 * | 0.70 * |
Average | 0.52 | 0.61 | 0.63 | 0.66 | 0.67 | 0.70 | 0.72 | 0.73 |
Multitask | RN50 | RF | RN50 | RF | RN50 | RF | RN50 | RF |
H/D | 0.64 | 0.55 | 0.16 * | 0.67 * | 0.96 * | 0.51 * | 0.70 | 0.74 |
Foveal scan | 0.53 * | 0.95 * | 0.91 | 0.94 | 0.50 * | 0.95 * | 0.75 * | 0.96 * |
Drusen | 0.52 | 0.55 | 0.32 * | 0.67 * | 0.96 * | 0.51 * | 0.70 | 0.64 |
PED | 0.70 | 0.72 | 0.44 * | 0.67 * | 0.72 | 0.73 | 0.58 | 0.71 |
H-Dots | 0.54 | 0.63 | 0.49 * | 0.67 * | 0.63 | 0.56 | 0.60 | 0.69 |
Average | 0.61 | 0.68 | 0.46 * | 0.73 * | 0.75 * | 0.64 * | 0.67 | 0.75 |
External (Kermany) Dataset | ||||||||
Single task | RN50 | RF | RN50 | RF | RN50 | RF | RN50 | RF |
H/D | 0.84 | 0.89 | 0.82 | 0.91 | 0.81 | 0.88 | 0.92 | 0.95 |
Drusen | 0.50 * | 0.93 * | 0.38 * | 0.92 * | 0.53 * | 0.94 * | 0.44 * | 0.84 * |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Du, K.; Nair, A.R.; Shah, S.; Gadari, A.; Vupparaboina, S.C.; Bollepalli, S.C.; Sutharahan, S.; Sahel, J.-A.; Jana, S.; Chhablani, J.; et al. Detection of Disease Features on Retinal OCT Scans Using RETFound. Bioengineering 2024, 11, 1186. https://doi.org/10.3390/bioengineering11121186
Du K, Nair AR, Shah S, Gadari A, Vupparaboina SC, Bollepalli SC, Sutharahan S, Sahel J-A, Jana S, Chhablani J, et al. Detection of Disease Features on Retinal OCT Scans Using RETFound. Bioengineering. 2024; 11(12):1186. https://doi.org/10.3390/bioengineering11121186
Chicago/Turabian StyleDu, Katherine, Atharv Ramesh Nair, Stavan Shah, Adarsh Gadari, Sharat Chandra Vupparaboina, Sandeep Chandra Bollepalli, Shan Sutharahan, José-Alain Sahel, Soumya Jana, Jay Chhablani, and et al. 2024. "Detection of Disease Features on Retinal OCT Scans Using RETFound" Bioengineering 11, no. 12: 1186. https://doi.org/10.3390/bioengineering11121186
APA StyleDu, K., Nair, A. R., Shah, S., Gadari, A., Vupparaboina, S. C., Bollepalli, S. C., Sutharahan, S., Sahel, J.-A., Jana, S., Chhablani, J., & Vupparaboina, K. K. (2024). Detection of Disease Features on Retinal OCT Scans Using RETFound. Bioengineering, 11(12), 1186. https://doi.org/10.3390/bioengineering11121186