Abdominal MRI Unconditional Synthesis with Medical Assessment
Abstract
:1. Introduction
1.1. Data Augmentation
1.2. Generative Adversarial Networks
1.3. Medical Image Synthesis with GANs
1.4. Evaluation of Synthetic Datasets
1.5. Related Work
1.6. Goals
2. Materials and Methods
2.1. Dataset Description and Preparation
2.2. Model Selection and Training
2.3. Quality Assessment
3. Results
3.1. Training Monitoring and Quantitative Evaluation
- As suggested in [33], the R1 gamma value was tailored to the training dataset’s image resolution and batch size.
- For smaller datasets with fewer than 30k images, it is advised to keep the adaptive augmentations (ADA) enabled.
3.2. Visual Inspection
3.3. Medical Assessment
4. Discussion
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Anaya-Isaza, A.; Mera-Jiménez, L.; Zequera-Diaz, M. An overview of deep learning in medical imaging. Inform. Med. Unlocked 2021, 26, 100723. [Google Scholar] [CrossRef]
- Elyan, E.; Vuttipittayamongkol, P.; Johnston, P.; Martin, K.; Sarker, M.M.K. Computer vision and machine learning for medical image analysis: Recent advances, challenges, and way forward. Artif. Intell. Surg. 2022, 2, 24–45. [Google Scholar] [CrossRef]
- Goceri, E. Medical image data augmentation: Techniques, comparisons and interpretations. Artif. Intell. Rev. 2023, 56, 12561–12605. [Google Scholar] [CrossRef] [PubMed]
- Chlap, P.; Min, H.; Vandenberg, N.; Dowling, J.; Holloway, L.; Haworth, A. A review of medical image data augmentation techniques for deep learning applications. J. Med. Imaging Radiat. Oncol. 2021, 65, 545–563. [Google Scholar] [CrossRef] [PubMed]
- Garcea, F.; Serra, A.; Lamberti, F.; Morra, L. Data augmentation for medical imaging: A systematic literature review. Comput. Biol. Med. 2023, 152, 106391. [Google Scholar] [CrossRef] [PubMed]
- Kazerouni, A.; Aghdam, E.K.; Heidari, M.; Azad, R.; Fayyaz, M.; Hacihaliloglu, I.; Merhof, D. Diffusion models in medical imaging: A comprehensive survey. Med. Image Anal. 2023, 88, 102846. [Google Scholar] [CrossRef] [PubMed]
- Shamshad, F.; Khan, S.; Zamir, S.W.; Khan, M.H.; Hayat, M.; Khan, F.S.; Fu, H. Transformers in medical imaging: A survey. Med. Image Anal. 2023, 88, 102802. [Google Scholar] [CrossRef] [PubMed]
- Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial Networks. arXiv 2014. [Google Scholar] [CrossRef]
- Skandarani, Y.; Jodoin, P.-M.; Lalande, A. GANs for Medical Image Synthesis: An Empirical Study. J. Imaging Sci. Technol. 2023, 9, 69. [Google Scholar] [CrossRef]
- Park, S.-W.; Ko, J.-S.; Huh, J.-H.; Kim, J.-C. Review on Generative Adversarial Networks: Focusing on Computer Vision and Its Applications. Electronics 2021, 10, 1216. [Google Scholar] [CrossRef]
- Yi, X.; Walia, E.; Babyn, P. Generative adversarial network in medical imaging: A review. Med. Image Anal. 2019, 58, 101552. [Google Scholar] [CrossRef] [PubMed]
- Borji, A. Pros and cons of GAN evaluation measures. Comput. Vis. Image Underst. 2019, 179, 41–65. [Google Scholar] [CrossRef]
- Karras, T.; Aittala, M.; Hellsten, J.; Laine, S.; Lehtinen, J.; Aila, T. Training Generative Adversarial Networks with Limited Data. arXiv 2020. [Google Scholar] [CrossRef]
- Tronchin, L.; Sicilia, R.; Cordelli, E.; Ramella, S.; Soda, P. Evaluating GANs in Medical Imaging. In Deep Generative Models, and Data Augmentation, Labelling, and Imperfections; Springer International Publishing: Berlin/Heidelberg, Germany, 2021; pp. 112–121. [Google Scholar] [CrossRef]
- Kazeminia, S.; Baur, C.; Kuijper, A.; van Ginneken, B.; Navab, N.; Albarqouni, S.; Mukhopadhyay, A. GANs for medical image analysis. Artif. Intell. Med. 2020, 109, 101938. [Google Scholar] [CrossRef] [PubMed]
- Sorin, V.; Barash, Y.; Konen, E.; Klang, E. Creating Artificial Images for Radiology Applications Using Generative Adversarial Networks (GANs)—A Systematic Review. Acad. Radiol. 2020, 27, 1175–1185. [Google Scholar] [CrossRef] [PubMed]
- Osuala, R.; Kushibar, K.; Garrucho, L.; Linardos, A.; Szafranowska, Z.; Klein, S.; Glocker, B.; Díaz, O.; Lekadir, K. A Review of Generative Adversarial Networks in Cancer Imaging: New Applications, New Solutions. arXiv 2021. Available online: https://www.semanticscholar.org/paper/35d2f0eb6e3c2ff7f61434231f4a59c1f4c9a49b (accessed on 12 May 2024).
- Frid-Adar, M.; Diamant, I.; Klang, E.; Amitai, M.; Goldberger, J.; Greenspan, H. GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification. arXiv 2018. [Google Scholar] [CrossRef]
- Fetty, L.; Bylund, M.; Kuess, P.; Heilemann, G.; Nyholm, T.; Georg, D.; Löfstedt, T. Latent space manipulation for high-resolution medical image synthesis via the StyleGAN. Z. Med. Phys. 2020, 30, 305–314. [Google Scholar] [CrossRef] [PubMed]
- Gonçalves, G.M. A Comparative Study of Data Augmentation Techniques for Image Classification: Generative Models vs. Classical Transformations. 2020. Available online: https://ria.ua.pt/handle/10773/30759 (accessed on 12 May 2024).
- Han, C.; Hayashi, H.; Rundo, L.; Araki, R.; Shimoda, W.; Muramatsu, S.; Furukawa, Y.; Mauri, G.; Nakayama, H. GAN-based synthetic brain MR image generation. In Proceedings of the 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018), Washington, DC, USA, 4–7 April 2018; IEEE: Piscataway, NJ, USA; pp. 734–738. [Google Scholar] [CrossRef]
- Kazuhiro, K.; Werner, R.A.; Toriumi, F.; Javadi, M.S.; Pomper, M.G.; Solnes, L.B.; Verde, F.; Higuchi, T.; Rowe, S.P. Generative Adversarial Networks for the Creation of Realistic Artificial Brain Magnetic Resonance Images. Tomography 2018, 4, 159–163. [Google Scholar] [CrossRef] [PubMed]
- Chuquicusma, M.J.M.; Hussein, S.; Burt, J.; Bagci, U. How to Fool Radiologists with Generative Adversarial Networks? A Visual Turing Test for Lung Cancer Diagnosis. arXiv 2017. [Google Scholar] [CrossRef]
- Korkinof, D.; Harvey, H.; Heindl, A.; Karpati, E.; Williams, G.; Rijken, T.; Kecskemethy, P.; Glocker, B. Perceived Realism of High-Resolution Generative Adversarial Network-derived Synthetic Mammograms. Radiol Artif. Intell. 2021, 3, e190181. [Google Scholar] [CrossRef]
- Levine, A.B.; Peng, J.; Farnell, D.; Nursey, M.; Wang, Y.; Naso, J.R.; Ren, H.; Farahani, H.; Chen, C.; Chiu, D.; et al. Synthesis of diagnostic quality cancer pathology images by generative adversarial networks. J. Pathol. 2020, 252, 178–188. [Google Scholar] [CrossRef] [PubMed]
- Ghorbani, A.; Natarajan, V.; Coz, D.; Liu, Y. DermGAN: Synthetic Generation of Clinical Skin Images with Pathology. In Proceedings of the Machine Learning for Health NeurIPS Workshop; Dalca, A.V., McDermott, M.B.A., Alsentzer, E., Finlayson, S.G., Oberst, M., Falck, F., Beaulieu-Jones, B., Eds.; PMLR: Cambridge, UK, 2020; Volume 116, pp. 155–170. Available online: https://proceedings.mlr.press/v116/ghorbani20a.html (accessed on 23 February 2024).
- Woodland, M.; Wood, J.; Anderson, B.M.; Kundu, S.; Lin, E.; Koay, E.; Odisio, B.; Chung, C.; Kang, H.C.; Venkatesan, A.M.; et al. Evaluating the Performance of StyleGAN2-ADA on Medical Images. arXiv 2022. [Google Scholar] [CrossRef]
- Gonçalves, B.; Vieira, P.; Vieira, A. Abdominal MRI Synthesis using StyleGAN2-ADA. In Proceedings of the 2023 IST-Africa Conference (IST-Africa), Tshwane, South Africa, 31 May–2 June 2023; IEEE: Piscataway, NJ, USA, 2023; pp. 1–9. [Google Scholar] [CrossRef]
- Karras, T.; Laine, S.; Aila, T. A Style-Based Generator Architecture for Generative Adversarial Networks. IEEE Trans. Pattern Anal. Mach. Intell. 2021, 43, 4217–4228. [Google Scholar] [CrossRef] [PubMed]
- Karras, T.; Aila, T.; Laine, S.; Lehtinen, J. Progressive Growing of GANs for Improved Quality, Stability, and Variation. arXiv 2017. [Google Scholar] [CrossRef]
- Gatys, L.A.; Ecker, A.S.; Bethge, M. A Neural Algorithm of Artistic Style. arXiv 2015. [Google Scholar] [CrossRef]
- Karras, T.; Laine, S.; Aittala, M.; Hellsten, J.; Lehtinen, J.; Aila, T. Analyzing and Improving the Image Quality of StyleGAN. arXiv 2019. [Google Scholar] [CrossRef]
- Karras, T.; Aittala, M.; Laine, S.; Härkönen, E.; Hellsten, J.; Lehtinen, J.; Aila, T. Alias-Free Generative Adversarial Networks. arXiv 2021. [Google Scholar] [CrossRef]
Attributes | Values |
---|---|
Gender | (number of people) |
Female | 104 |
Male | 103 |
Total (4 undefined) | 211 |
Median age (years) | 65 |
Lesion type | (% of total lesions) |
Adenoma | 82.50 |
Metastasis | 12.80 |
Pheochromocytoma | 2.80 |
Carcinoma | 0.90 |
Myelolipoma | 0.50 |
Lymphoma | 0.50 |
Hyperparameter | Value |
---|---|
Configuration | stylegan3-r |
Gamma | 8.2 |
Kimgs | 2000 |
Augmentation mode | ADA |
Number of GPUs | 1 |
Batch size | 4 |
Snap | 200 |
Evaluation Metric | Value |
---|---|
FID against the full dataset | 12.89 |
KID against the full dataset | 7.06 × 10−3 |
Radiologist Identified as Synthetic | Radiologist Identified as Real | Total | |
---|---|---|---|
Synthetic image set | 4 | 17 | 21 |
Real image set | 5 | 10 | 15 |
Total | 9 | 27 | 36 |
Characteristics | Degree of Relevance (1–4) |
---|---|
Visual artefacts (irregular edges, distortion areas) | 3 |
Inconsistencies in anatomy | 4 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Gonçalves, B.; Silva, M.; Vieira, L.; Vieira, P. Abdominal MRI Unconditional Synthesis with Medical Assessment. BioMedInformatics 2024, 4, 1506-1518. https://doi.org/10.3390/biomedinformatics4020082
Gonçalves B, Silva M, Vieira L, Vieira P. Abdominal MRI Unconditional Synthesis with Medical Assessment. BioMedInformatics. 2024; 4(2):1506-1518. https://doi.org/10.3390/biomedinformatics4020082
Chicago/Turabian StyleGonçalves, Bernardo, Mariana Silva, Luísa Vieira, and Pedro Vieira. 2024. "Abdominal MRI Unconditional Synthesis with Medical Assessment" BioMedInformatics 4, no. 2: 1506-1518. https://doi.org/10.3390/biomedinformatics4020082
APA StyleGonçalves, B., Silva, M., Vieira, L., & Vieira, P. (2024). Abdominal MRI Unconditional Synthesis with Medical Assessment. BioMedInformatics, 4(2), 1506-1518. https://doi.org/10.3390/biomedinformatics4020082