Using Deep Learning Methods for Segmenting Polar Mesospheric Summer Echoes
Abstract
:1. Introduction
2. Theory
2.1. UNet Architectures
2.2. Evaluation Metrics
2.2.1. Jaccard Index
2.2.2. Dice–Sørensen Coefficient
2.3. Loss Function
2.3.1. Binary Cross Entropy
2.3.2. Dice Loss
2.3.3. Focal Loss
2.3.4. Boundary Loss
2.3.5. Dice–BCE Loss
2.3.6. Dice–Boundary Loss
2.4. Data Augmentation
Image-Level Augmentation
2.5. Object-Level Augmentation
3. Data
3.1. Constructing Samples from Data
3.2. Data Augmentation Procedure
4. Model Hyperparameters
5. Results
5.1. Initial Experiment
- Randomly initiated weights with 32 and 64 initial feature maps.
- Pretrained weight initiation of the encoder layers. For the models with 32 initial feature maps, a pretrained UNet model found at (Note: https://pytorch.org/hub/mateuszbuda_brain-segmentation-pytorch_unet, accessed on 14 August 2023) is used. For the models with 64 initial feature maps, a VGG16 [31] model pretrained on ImageNet [27] is used as the backbone.
5.2. Using Different Loss Functions
- : is set equal to that of thr original paper [18], while is chosen such that it is approximately inversely proportional to the foreground frequency.
- –
- Increase: For the increase schedule, initially, and it is increased by every five iterations, where .
- –
- Rebalance: For the Rebalance, initially and follows a schedule based on the number of iterations as followsThis is a slightly different scheduling of than that of the original rebalance strategy [20] and is considered necessary as the model struggles when is increased too quickly in the start.
5.3. Using Image-Level and Object-Level Augmentations
6. Discussion
7. Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- Ecklund, W.L.; Balsley, B.B. Long-term observations of the Arctic mesosphere with the MST radar at Poker Flat, Alaska. J. Geophys. Res. Space Phys. 1981, 86, 7775–7780. [Google Scholar] [CrossRef]
- Rapp, M.; Lübken, F.J. Polar mesosphere summer echoes (PMSE): Review of observations and current understanding. Atmos. Chem. Phys. 2004, 4, 2601–2633. [Google Scholar] [CrossRef]
- Latteck, R.; Renkwitz, T.; Chau, J.L. Two decades of long-term observations of polar mesospheric echoes at 69°N. J. Atmos. Sol.-Terr. Phys. 2021, 216, 105576. [Google Scholar] [CrossRef]
- Gunnarsdottir, T.L.; Poggenpohl, A.; Mann, I.; Mahmoudian, A.; Dalin, P.; Haeggstroem, I.; Rietveld, M. Modulation of polar mesospheric summer echoes (PMSEs) with high-frequency heating during low solar illumination. Ann. Geophys. 2023, 41, 93–114. [Google Scholar] [CrossRef]
- Mann, I.; Häggström, I.; Tjulin, A.; Rostami, S.; Anyairo, C.C.; Dalin, P. First wind shear observation in PMSE with the tristatic EISCAT VHF radar. J. Geophys. Res. Space Phys. 2016, 121, 11271–11281. [Google Scholar] [CrossRef]
- EISCAT Scientific Association. Available online: http://eiscat.se (accessed on 5 August 2023).
- Jozwicki, D.; Sharma, P.; Mann, I. Investigation of Polar Mesospheric Summer Echoes Using Linear Discriminant Analysis. Remote Sens. 2021, 13, 522. [Google Scholar] [CrossRef]
- Jozwicki, D.; Sharma, P.; Mann, I.; Hoppe, U.P. Segmentation of PMSE Data Using Random Forests. Remote Sens. 2022, 14, 2976. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Munich, Germany, 5–9 October 2015; Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar]
- Zhou, Z.; Rahman Siddiquee, M.M.; Tajbakhsh, N.; Liang, J. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. In Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Granada, Spain, 20 September 2018; Stoyanov, D., Taylor, Z., Carneiro, G., Syeda-Mahmood, T., Martel, A., Maier-Hein, L., Tavares, J.M.R., Bradley, A., Papa, J.P., Belagiannis, V., et al., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 3–11. [Google Scholar]
- Long, J.; Shelhamer, E.; Darrell, T. Fully Convolutional Networks for Semantic Segmentation. In Proceedings of the The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015. [Google Scholar]
- Wikipedia Contributors. Jaccard Index—Wikipedia, The Free Encyclopedia. 2023. Available online: https://en.wikipedia.org/wiki/Jaccard_index (accessed on 11 March 2023).
- Dice, L.R. Measures of the Amount of Ecologic Association Between Species. Ecology 1945, 26, 297–302. [Google Scholar] [CrossRef]
- Janocha, K.; Czarnecki, W. On Loss Functions for Deep Neural Networks in Classification. Schedae Inform. 2017, 25. [Google Scholar] [CrossRef]
- Jadon, S. A survey of loss functions for semantic segmentation. In Proceedings of the 2020 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), Via del Mar, Chile, 27–29 October 2020. [Google Scholar] [CrossRef]
- Yi-de, M.; Qing, L.; Zhi-bai, Q. Automated image segmentation using improved PCNN model based on cross-entropy. In Proceedings of the 2004 International Symposium on Intelligent Multimedia, Video and Speech Processing, Hong Kong, China, 20–22 October 2004; pp. 743–746. [Google Scholar] [CrossRef]
- Sudre, C.H.; Li, W.; Vercauteren, T.; Ourselin, S.; Jorge Cardoso, M. Generalised Dice Overlap as a Deep Learning Loss Function for Highly Unbalanced Segmentations. In Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Quebec City, QC, Canada, 14 September 2017; Cardoso, M.J., Arbel, T., Carneiro, G., Syeda-Mahmood, T., Tavares, J.M.R., Moradi, M., Bradley, A., Greenspan, H., Papa, J.P., Madabhushi, A., et al., Eds.; Springer International Publishing: Cham, Switzerland, 2017; pp. 240–248. [Google Scholar]
- Lin, T.Y.; Goyal, P.; Girshick, R.; He, K.; Dollár, P. Focal Loss for Dense Object Detection. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 22–29 October 2017; pp. 2999–3007. [Google Scholar] [CrossRef]
- Boykov, Y.; Kolmogorov, V.; Cremers, D.; Delong, A. An Integral Solution to Surface Evolution PDEs Via Geo-cuts. In Proceedings of the Computer Vision—ECCV 2006, Graz, Austria, 7–13 May 2006; Springer: Berlin/Heidelberg, Germany, 2006. Lecture Notes in Computer Science. pp. 409–422. [Google Scholar]
- Kervadec, H.; Bouchtiba, J.; Desrosiers, C.; Granger, E.; Dolz, J.; Ayed, I.B. Boundary loss for highly unbalanced segmentation. Med. Image Anal. 2021, 67, 101851. [Google Scholar] [CrossRef] [PubMed]
- Ayan, E.; Ünver, H.M. Data augmentation importance for classification of skin lesions via deep learning. In Proceedings of the 2018 Electric Electronics, Computer Science, Biomedical Engineerings’ Meeting (EBBT), Istanbul, Turkey, 18–19 April 2018; pp. 1–4. [Google Scholar] [CrossRef]
- Zhang, J.; Zhang, Y.; Xu, X. ObjectAug: Object-level Data Augmentation for Semantic Image Segmentation. In Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China, 18–22 July 2021; pp. 1–8. [Google Scholar] [CrossRef]
- Shorten, C.; Khoshgoftaar, T.M. A survey on Image Data Augmentation for Deep Learning. J. Big Data 2019, 6, 1–48. [Google Scholar] [CrossRef]
- Lehtinen, M.S.; Huuskonen, A. General incoherent scatter analysis and GUISDAP. J. Atmos. Terr. Phys. 1996, 58, 435–452. [Google Scholar] [CrossRef]
- Liu, G.; Reda, F.A.; Shih, K.J.; Wang, T.; Tao, A.; Catanzaro, B. Image Inpainting for Irregular Holes Using Partial Convolutions. In Proceedings of the Computer Vision—ECCV 2018—15th European Conference, Munich, Germany, 8–14 September 2018; Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y., Eds.; Springer: Berlin/Heidelberg, Germany, 2018; Volume 11215, pp. 89–105. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar] [CrossRef]
- Deng, J.; Dong, W.; Socher, R.; Li, L.J.; Li, K.; Fei-Fei, L. ImageNet: A large-scale hierarchical image database. In Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, 20–25 June 2009; pp. 248–255. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile, 7–13 December 2015; Volume 1502. [Google Scholar] [CrossRef]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. In Proceedings of the ICLR (Poster), San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- Goodfellow, I.J.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- Montavon, G.; Binder, A.; Lapuschkin, S.; Samek, W.; Müller, K.R. Layer-Wise Relevance Propagation: An Overview; Springer: Cham, Switzerland, 2019; pp. 193–209. [Google Scholar] [CrossRef]
Model–Initiation | Hyperparameters | |
---|---|---|
Learning Rate | Weight Decay | |
UNet–RandomInit | 0.008 | 0.005 |
UNet–Pretrained | 0.003 | 0.007 |
UNet–RandomInit | 0.006 | 0.005 |
UNet–Pretrained | 0.003 | 0.007 |
UNet++–RandomInit | 0.005 | 0.005 |
UNet++–Pretrained | 0.003 | 0.006 |
UNet++–RandomInit | 0.002 | 0.006 |
UNet++–Pretrained | 0.001 | 0.008 |
Model–Weight Initiation | Test | Validation | ||
---|---|---|---|---|
IoU | DSC | IoU | DSC | |
UNet–RandomInit | 0.654 ± 0.006 | 0.791 ± 0.005 | 0.710 ± 0.007 | 0.830 ± 0.005 |
UNet–Pretrained | 0.634 ± 0.010 | 0.776 ± 0.007 | 0.699 ± 0.008 | 0.823 ± 0.006 |
UNet–RandomInit | 0.649 ± 0.005 | 0.787 ± 0.003 | 0.713 ± 0.011 | 0.832 ± 0.008 |
UNet–Pretrained | 0.645 ± 0.005 | 0.784 ± 0.004 | 0.702 ± 0.005 | 0.825 ± 0.003 |
UNet++–RandomInit | 0.654 ± 0.012 | 0.790 ± 0.008 | 0.713 ± 0.005 | 0.833 ± 0.003 |
UNet++–Pretrained | 0.632 ± 0.027 | 0.774 ± 0.021 | 0.692 ± 0.030 | 0.817 ± 0.021 |
UNet++–RandomInit | 0.666 ± 0.010 | 0.799 ± 0.007 | 0.727 ± 0.008 | 0.842 ± 0.005 |
UNet++–Pretrained | 0.649 ± 0.006 | 0.787 ± 0.004 | 0.719 ± 0.008 | 0.837 ± 0.005 |
Loss Function | Test | Validation | ||
---|---|---|---|---|
IoU | DSC | IoU | DSC | |
0.666 ± 0.010 | 0.799 ± 0.007 | 0.727 ± 0.008 | 0.842 ± 0.005 | |
0.656 ± 0.006 | 0.792 ± 0.004 | 0.714 ± 0.003 | 0.833 ± 0.002 | |
0.647 ± 0.003 | 0.786 ± 0.002 | 0.695 ± 0.002 | 0.820 ± 0.001 | |
0.667 ± 0.005 | 0.800 ± 0.003 | 0.731 ± 0.004 | 0.844 ± 0.003 | |
–Increase | 0.662 ± 0.013 | 0.797 ± 0.010 | 0.722 ± 0.011 | 0.838 ± 0.007 |
–Rebalance | 0.650 ± 0.011 | 0.788 ± 0.008 | 0.703 ± 0.012 | 0.825 ± 0.008 |
Model–Augmentation | Test | Validation | ||
---|---|---|---|---|
IoU | DSC | IoU | DSC | |
Baseline | 0.667 ± 0.005 | 0.800 ± 0.003 | 0.731 ± 0.004 | 0.844 ± 0.003 |
Horizontal Flip | 0.682 ± 0.010 | 0.811 ± 0.007 | 0.742 ± 0.008 | 0.851 ± 0.005 |
Vertical Flip | 0.672 ± 0.006 | 0.804 ± 0.004 | 0.739 ± 0.009 | 0.849 ± 0.006 |
Contrast Adjust | 0.683 ± 0.005 | 0.811 ± 0.003 | 0.742 ± 0.002 | 0.851 ± 0.001 |
All Combined | 0.669 ± 0.007 | 0.801 ± 0.005 | 0.741 ± 0.008 | 0.851 ± 0.006 |
Horizontal and Contrast Adjust | 0.694 ± 0.008 | 0.819 ± 0.006 | 0.735 ± 0.004 | 0.847 ± 0.003 |
UNet++–RandomInit | Test | Validation | ||
---|---|---|---|---|
IoU | DSC | IoU | DSC | |
No Aug | 0.667 ± 0.005 | 0.800 ± 0.003 | 0.731 ± 0.004 | 0.844 ± 0.003 |
Image-Aug | 0.694 ± 0.008 | 0.819 ± 0.006 | 0.735 ± 0.004 | 0.847 ± 0.003 |
ObjAug | 0.678 ± 0.009 | 0.808 ± 0.007 | 0.719 ± 0.007 | 0.836 ± 0.005 |
ObjAug and Image-Aug | 0.701 ± 0.010 | 0.824 ± 0.007 | 0.730 ± 0.003 | 0.843 ± 0.002 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Domben, E.S.; Sharma, P.; Mann, I. Using Deep Learning Methods for Segmenting Polar Mesospheric Summer Echoes. Remote Sens. 2023, 15, 4291. https://doi.org/10.3390/rs15174291
Domben ES, Sharma P, Mann I. Using Deep Learning Methods for Segmenting Polar Mesospheric Summer Echoes. Remote Sensing. 2023; 15(17):4291. https://doi.org/10.3390/rs15174291
Chicago/Turabian StyleDomben, Erik Seip, Puneet Sharma, and Ingrid Mann. 2023. "Using Deep Learning Methods for Segmenting Polar Mesospheric Summer Echoes" Remote Sensing 15, no. 17: 4291. https://doi.org/10.3390/rs15174291