Semi-Supervised Learning in Medical MRI Segmentation: Brain Tissue with White Matter Hyperintensity Segmentation Using FLAIR MRI
Abstract
:1. Introduction
2. Materials and Methods
2.1. Subjects
2.2. Overview of the Proposed Method
2.3. Brain Tissue Segmentation from FLAIR MRI: Pseudo-Labeling-Based Segmentation
2.3.1. Pseudo-Labeling from T1w MRI
2.3.2. Co-Registration
2.4. Brain Tissue Segmentation Enhancement
2.4.1. Deep Learning-Based Initial Segmentation
2.4.2. Morphological Label Correction
2.5. WMH Segmentation: Supervised Learning-Based Segmentation
2.5.1. Annotated Labeling with Radiologists
2.5.2. Preprocessing
2.6. Training
- RandomAffine, which has a scale parameter in the range of 0.85–1.15.
- RandomMotion, with a degree value up to 10 and a translation value up to 10 mm.
- RandomBiasField, with a magnitude coefficient parameter ranging between −0.5 and 0.5.
- RandomNoise, which has a mean value of Gaussian distribution in range of 0 to 0.025.
- RandomFlip, with a spatial transform value up to 2, which inverts the Z axis.
2.7. Experiment Setup
2.8. Metrics for Evaluation
2.8.1. Evaluation for Brain Tissue Segmentation
2.8.2. Evaluation for WMH Segmentation
3. Results
3.1. Measured Volume Comparisons: Brain Tissue Segmentation
3.2. Dice Overlap Scores: WMH Segmentation
4. Discussion
4.1. Performance of Brain Tissue Segmentation
4.2. Performance of WMH Segmentation
4.3. Clinical Relevance and Application
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Caligiuri, M.E.; Perrotta, P.; Augimeri, A.; Rocca, F.; Quattrone, A.; Cherubini, A. Automatic Detection of White Matter Hyperintensities in Healthy Aging and Pathology Using Magnetic Resonance Imaging: A Review. Neuroinformatics 2015, 13, 261–276. [Google Scholar] [CrossRef] [Green Version]
- Guerrero, R.; Qin, C.; Oktay, O.; Bowles, C.; Chen, L.; Joules, R.; Wolz, R.; Valdés-Hernández, M.C.; Dickie, D.A.; Wardlaw, J.; et al. White matter hyperintensity and stroke lesion segmentation and differentiation using convolutional neural networks. Neuroimage Clin. 2018, 17, 918–934. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Hase, Y.; Horsburgh, K.; Ihara, M.; Kalaria, R.N. White matter degeneration in vascular and other ageing-related dementias. J. Neurochem. 2018, 144, 617–633. [Google Scholar] [CrossRef]
- Au, R.; Massaro, J.M.; Wolf, P.A.; Young, M.E.; Beiser, A.; Seshadri, S.; D’Agostino, R.B.; DeCarli, C. Association of white matter hyperintensity volume with decreased cognitive functioning: The Framingham Heart Study. Arch. Neurol. 2006, 63, 246–250. [Google Scholar] [CrossRef] [PubMed]
- Garnier-Crussard, A.; Bougacha, S.; Wirth, M.; André, C.; Delarue, M.; Landeau, B.; Mézenge, F.; Kuhn, E.; Gonneaud, J.; Chocat, A.; et al. White matter hyperintensities across the adult lifespan: Relation to age, Aβ load, and cognition. Alzheimer’s Res. Ther. 2020, 12, 1–11. [Google Scholar] [CrossRef] [PubMed]
- Prasad, K.; Wiryasaputra, L.; Ng, A.; Kandiah, N. White matter disease independently predicts progression from mild cognitive impairment to Alzheimer’s disease in a clinic cohort. Dement. Geriatr. Cogn. Disord. 2011, 31, 431–434. [Google Scholar] [CrossRef]
- Kearney-Schwartz, A.; Rossignol, P.; Bracard, S.; Felblinger, J.; Fay, R.; Boivin, J.M.; Lecompte, T.; Lacolley, P.; Benetos, A.; Zannad, F. Vascular structure and function is correlated to cognitive performance and white matter hyperintensities in older hypertensive patients with subjective memory complaints. Stroke 2009, 40, 1229–1236. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kim, S.J.; Lee, D.K.; Jang, Y.K.; Jang, H.; Kim, S.E.; Cho, S.H.; Kim, J.P.; Jung, Y.H.; Kim, E.J.; Na, D.L.; et al. The Effects of Longitudinal White Matter Hyperintensity Change on Cognitive Decline and Cortical Thinning over Three Years. J. Clin. Med. 2020, 9, 2663. [Google Scholar] [CrossRef]
- Fischl, B. FreeSurfer. NeuroImage 2012, 62, 774–781. [Google Scholar] [CrossRef] [Green Version]
- Ashburner, J.; Friston, K.J. Unified segmentation. Neuroimage 2005, 26, 839–851. [Google Scholar] [CrossRef]
- Jenkinson, M.; Beckmann, C.F.; Behrens, T.E.; Woolrich, M.W.; Smith, S.M. Fsl. NeuroImage 2012, 62, 782–790. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Perkuhn, M.; Stavrinou, P.; Thiele, F.; Shakirin, G.; Mohan, M.; Garmpis, D.; Kabbasch, C.; Borggrefe, J. Clinical Evaluation of a Multiparametric Deep Learning Model for Glioblastoma Segmentation Using Heterogeneous Magnetic Resonance Imaging Data from Clinical Routine. Investig. Radiol. 2018, 53, 647–654. [Google Scholar] [CrossRef] [PubMed]
- Zhu, X.; Goldberg, A.B. Introduction to semi-supervised learning. Synth. Lect. Artif. Intell. Mach. Learn. 2009, 3, 1–130. [Google Scholar] [CrossRef] [Green Version]
- Beare, R.; Lowekamp, B.; Yaniv, Z. Image Segmentation, Registration and Characterization in R with SimpleITK. J. Stat. Softw. 2018, 86. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Zhou, Z.; Rahman Siddiquee, M.M.; Tajbakhsh, N.; Liang, J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support; Stoyanov, D., Taylor, Z., Carneiro, G., Syeda-Mahmood, T., Martel, A., Maier-Hein, L., Tavares, J.M.R.S., Bradley, A., Papa, J.P., Belagiannis, V., et al., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 3–11. [Google Scholar]
- Liu, H.; Brock, A.; Simonyan, K.; Le, Q.V. Evolving normalization-activation layers. arXiv 2020, arXiv:2004.02967. [Google Scholar]
- Pérez-García, F.; Sparks, R.; Ourselin, S. TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. arXiv 2020, arXiv:2003.04696. [Google Scholar]
- Lehmann, G. Label object representation and manipulation with ITK. Insight J. 2007, 8, 1–31. [Google Scholar]
- Isensee, F.; Schell, M.; Pflueger, I.; Brugnara, G.; Bonekamp, D.; Neuberger, U.; Wick, A.; Schlemmer, H.P.; Heiland, S.; Wick, W.; et al. Automated brain extraction of multisequence MRI using artificial neural networks. Hum. Brain Mapp. 2019, 40, 4952–4964. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Li, W.; Wang, G.; Fidon, L.; Ourselin, S.; Cardoso, M.J.; Vercauteren, T. On the compactness, efficiency, and representation of 3D convolutional networks: Brain parcellation as a pretext task. In Proceedings of the International Conference on Information Processing in Medical Imaging, Boone, NC, USA, 25–30 June 2017; pp. 348–360. [Google Scholar] [CrossRef] [Green Version]
- Paszke, A.; Gross, S.; Massa, F.; Lerer, A.; Bradbury, J.; Chanan, G.; Killeen, T.; Lin, Z.; Gimelshein, N.; Antiga, L.; et al. Pytorch: An imperative style, high-performance deep learning library. arXiv 2019, arXiv:1912.01703. [Google Scholar]
- Rubinstein, R.Y.; Kroese, D.P. The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning; Springer: New York, NY, USA, 2013. [Google Scholar]
- Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv 2014, arXiv:1412.6980. [Google Scholar]
- Sudre, C.H.; Li, W.; Vercauteren, T.; Ourselin, S.; Cardoso, M.J. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support; Springer: Cham, Switzerland, 2017; pp. 240–248. [Google Scholar]
- Sorensen, T.A. A method of establishing groups of equal amplitude in plant sociology based on similarity of species content and its application to analyses of the vegetation on Danish commons. Biol. Skar. 1948, 5, 1–34. [Google Scholar]
Dataset | MRI | No. of Subjects | Matrix Size | Pixel Spacing (mm) | Purpose |
---|---|---|---|---|---|
CABI | T1w | 68 | 256 × 256 × 256 | 1.0 × 1.0 × 1.0 | Brain tissue segmentation |
CABI | FLAIR | 68 | 348 × 384 × 28 | 0.57 × 0.57 × 6 | Brain tissue segmentation |
CABI | FLAIR | 308 | 768 × 768 × 32 | 0.27 × 0.27 × 5 | WMH segmentation |
Measurement | Brain Tissue | FreeSurfer Label (T1w) | Pseudo Label (FLAIR) | U-Net++ | U-Net | HighRes3DNet |
---|---|---|---|---|---|---|
Volume (mL, mean ± SD) | Cerebellum GM | 430.8 ± 45.7 | 444.5 ± 47.3 | 458.0 ± 45.4 | 455.4 ± 42.0 | 408.0 ± 34.6 |
Cerebellum WM | 499.3 ± 55.8 | 516.3 ± 57.5 | 510.4 ± 52.7 | 519.5 ± 54.4 | 559.3 ± 56.4 | |
Cerebrum GM | 100.2 ± 10.2 | 103.5 ± 10.7 | 102.2 ± 9.6 | 101.1 ± 9.6 | 96.7 ± 8.9 | |
Cerebrum WM | 23.2 ± 3.0 | 24.0 ± 3.1 | 22.5 ± 3.1 | 23.7 ± 2.8 | 22.9 ± 3.6 | |
Lateral Ventricles | 41.2 ± 20.8 | 42.5 ± 21.6 | 39.1 ± 20.6 | 39.9 ± 20.7 | 41.5 ± 21.3 |
Measurement | Brain Tissue | Pseudo Label (FLAIR) | U-Net++ | U-Net | HighRes3DNet |
---|---|---|---|---|---|
Relative Difference (%, mean ± SD) | Cerebellum GM | 3.2 ± 1.1 | 6.4 ± 2.5 | 5.9 ± 2.8 | 5.4 ± 4.0 |
Cerebellum WM | 3.4 ± 0.9 | 2.5 ± 2.0 | 4.1 ± 2.0 | 12.2 ± 3.6 | |
Cerebrum GM | 3.3 ± 1.4 | 2.7 ± 2.2 | 2.3 ± 1.7 | 4.3 ± 3.1 | |
Cerebrum WM | 4.3 ± 3.2 | 6.1 ± 3.7 | 6.8 ± 6.7 | 9.7 ± 7.9 | |
Lateral Ventricles | 3.1 ± 1.4 | 6.1 ± 4.3 | 4.7 ± 4.3 | 5.4 ± 6.0 | |
Average Difference (%, mean ± SD) | - | 3.4 ± 0.5 | 4.8 ± 2.0 | 4.8 ± 1.7 | 7.4 ± 3.4 |
Dice Overlap Score | Precision | Recall | F1 Score | |
---|---|---|---|---|
U-Net++ | 0.77 ± 0.09 | 0.88 ± 0.05 | 0.80 ± 0.08 | 0.83 ± 0.05 |
U-Net | 0.81 ± 0.07 | 0.86 ± 0.06 | 0.84 ± 0.08 | 0.84 ± 0.04 |
HighRes3DNet | 0.73 ± 0.07 | 0.64 ± 0.09 | 0.92 ± 0.06 | 0.75 ± 0.07 |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Rieu, Z.; Kim, J.; Kim, R.E.; Lee, M.; Lee, M.K.; Oh, S.W.; Wang, S.-M.; Kim, N.-Y.; Kang, D.W.; Lim, H.K.; et al. Semi-Supervised Learning in Medical MRI Segmentation: Brain Tissue with White Matter Hyperintensity Segmentation Using FLAIR MRI. Brain Sci. 2021, 11, 720. https://doi.org/10.3390/brainsci11060720
Rieu Z, Kim J, Kim RE, Lee M, Lee MK, Oh SW, Wang S-M, Kim N-Y, Kang DW, Lim HK, et al. Semi-Supervised Learning in Medical MRI Segmentation: Brain Tissue with White Matter Hyperintensity Segmentation Using FLAIR MRI. Brain Sciences. 2021; 11(6):720. https://doi.org/10.3390/brainsci11060720
Chicago/Turabian StyleRieu, ZunHyan, JeeYoung Kim, Regina EY Kim, Minho Lee, Min Kyoung Lee, Se Won Oh, Sheng-Min Wang, Nak-Young Kim, Dong Woo Kang, Hyun Kook Lim, and et al. 2021. "Semi-Supervised Learning in Medical MRI Segmentation: Brain Tissue with White Matter Hyperintensity Segmentation Using FLAIR MRI" Brain Sciences 11, no. 6: 720. https://doi.org/10.3390/brainsci11060720