U-Net Model with Transfer Learning Model as a Backbone for Segmentation of Gastrointestinal Tract
Abstract
:1. Introduction
- The proposed U-Net model has been deployed with six pretrained transfer learning models as a backbone to analyse its performance. The six transfer learning models chosen for the backbone of U-Net are Inception V3, SeResNet50, VGG19, DenseNet121, InceptionResNetV2, and EfficientNet B0.
- This work proposed a U-Net model based on deep learning that has been created for the small size of images so that local features for segmentation can be enhanced and extracted efficiently.
- The proposed U-Net model has been deployed on the UW-Madison GI tract image segmentation dataset for the stomach, small bowel, and large bowel segmentation in the GI tract.
- Model performance metrics such as model loss, dice coefficient, and IoU coefficient are used to evaluate the models.
2. Related Work
3. Proposed Methodology
3.1. Input Dataset
3.2. Dataset Pre-Processing
3.2.1. Resizing
3.2.2. Gaussian Filter
3.2.3. Normalization
3.2.4. Augmentation
3.3. Segmentation Using Proposed U-Net Model
3.4. Segmentation Using Pre-Trained Transfer Learning Models
4. Results and Discussion
4.1. Hyperparameter Tuning
4.2. Analysis of Training and Validation Loss
4.3. Analysis of Dice Coefficient
4.4. Analysis of IoU Coefficient
4.5. Visual Analysis of Segmented Images
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Jaffray, D.A.; Gospodarowicz, M.K. Radiation therapy for cancer. Cancer Dis. Control. Prior. 2015, 3, 239–248. [Google Scholar]
- Rawla, P.; Barsouk, A. Epidemiology of gastric cancer: Global trends, risk factors and prevention. Gastroenterol. Rev. Prz. 2019, 14, 26–38. [Google Scholar] [CrossRef] [PubMed]
- Ahsan, H.; Neugut, A.I. Radiation therapy for breast cancer and increased risk for esophageal carcinoma. Ann. Intern. Med. 1998, 128, 114–117. [Google Scholar] [CrossRef] [PubMed]
- Lagendijk, J.J.; Raaymakers, B.W.; Van Vulpen, M. The magnetic resonance imaging–linac system. In Seminars in Radiation Oncology; Elsevier: Amsterdam, The Netherlands, 2014; Volume 24, pp. 207–209. [Google Scholar]
- Fan, S.; Xu, L.; Fan, Y.; Wei, K.; Li, L. Computer-aided detection of small intestinal ulcer and erosion in wireless capsule endoscopy images. Phys. Med. Biol. 2018, 63, 165001. [Google Scholar] [CrossRef]
- Khan, M.A.; Akram, T.; Sharif, M.; Javed, K.; Rashid, M.; Bukhari, S.A.C. An integrated framework of skin lesion detection and recognition through saliency method and optimal deep neural network features selection. Neural Comput. Appl. 2019, 32, 15929–15948. [Google Scholar] [CrossRef]
- Murugan, S.; Venkatesan, C.; Sumithra, M.G.; Gao, X.-Z.; Elakkiya, B.; Akila, M.; Manoharan, S. DEMNET: A deep learning model for early diagnosis of Alzheimer diseases and dementia from MR images. IEEE Access 2021, 9, 90319–90329. [Google Scholar] [CrossRef]
- Chandran, V.; Sumithra, M.G.; Karthick, A.; George, T.; Deivakani, M.; Elakkiya, B.; Subramaniam, U.; Manoharan, S. Diagnosis of cervical cancer based on ensemble deep learning network using colposcopy images. BioMed Res. Int. 2021, 2021, 5584004. [Google Scholar] [CrossRef]
- Khosla, A.; Khandnor, P.; Chand, T. A comparative analysis of signal processing and classification methods for different applications based on EEG signals. Biocybern. Biomed. Eng. 2020, 40, 649–690. [Google Scholar] [CrossRef]
- Tang, P.; Liang, Q.; Yan, X.; Xiang, S.; Sun, W.; Zhang, D.; Coppola, G. Efficient skin lesion segmentation using separable-Unet with stochastic weight averaging. Comput. Methods Programs Biomed. 2019, 178, 289–301. [Google Scholar] [CrossRef]
- Kalinin, A.A.; Iglovikov, V.I.; Rakhlin, A.; Shvets, A.A. Medical image segmentation using deep neural networks with pre-trained encoders. In Deep Learning Applications; Springer: Singapore, 2020; pp. 39–52. [Google Scholar]
- Ali, S.; Dmitrieva, M.; Ghatwary, N.; Bano, S.; Polat, G.; Temizel, A.; Krenzer, A.; Hekalo, A.; Guo, Y.B.; Rittscher, J.; et al. Deep learning for detection and segmentation of artefact and disease instances in gastrointestinal endoscopy. Med. Image Anal. 2021, 70, 102002. [Google Scholar] [CrossRef]
- Charfi, S.; El Ansari, M. Computer-aided diagnosis system for colon abnormalities detection in wireless capsule endoscopy images. Multimed. Tools Appl. 2018, 77, 4047–4064. [Google Scholar] [CrossRef]
- Charfi, S.; El Ansari, M.; Balasingham, I. Computer-aided diagnosis system for ulcer detection in wireless capsule endoscopy images. IET Image Process. 2019, 13, 1023–1030. [Google Scholar] [CrossRef] [Green Version]
- Souaidi, M.; El Ansari, M. Multi-scale analysis of ulcer disease detection from WCE images. IET Image Process. 2019, 13, 2233–2244. [Google Scholar] [CrossRef]
- Souaidi, M.; Abdelouahed, A.A.; El Ansari, M. Multi-scale completed local binary patterns for ulcer detection in wireless capsule endoscopy images. Multimed. Tools Appl. 2019, 78, 13091–13108. [Google Scholar] [CrossRef]
- Naqvi, S.S.A.; Nadeem, S.; Zaid, M.; Tahir, M.A. Ensemble of texture features for finding abnormalities in the gastro-intestinal tract. In Proceedings of the 2017 Multimedia Benchmark Workshop, MediaEval’17, Dublin, Ireland, 13–15 September 2017. [Google Scholar]
- Liu, Y.; Gu, Z.; Cheung, W.K. Hkbu at Mediaeval 2017 medico: Medical multimedia task. In Proceedings of the 2017 Multimedia Benchmark Workshop, MediaEval’17, Dublin, Ireland, 13–15 September 2017. [Google Scholar]
- Asperti, A.; Mastronardo, C. The effectiveness of data augmentation for detection of gastrointestinal diseases from endoscopical images. arXiv 2017, arXiv:1712.03689. [Google Scholar]
- Zhang, X.; Hu, W.; Chen, F.; Liu, J.; Yang, Y.; Wang, L.; Duan, H.; Si, J. Gastric precancerous diseases classification using CNN with a concise model. PLoS ONE 2017, 12, e0185508. [Google Scholar] [CrossRef]
- Agrawal, T.; Gupta, R.; Sahu, S.; Espy-Wilson, C.Y. SCL-UMD at the Medico Task-Mediaeval 2017: Transfer Learning Based Classification of Medical Images. In Proceedings of the Multimedia Benchmark Workshop, MediaEval’17, Dublin, Ireland, 13–15 September 2017. [Google Scholar]
- Pogorelov, K.; Randel, K.R.; Griwodz, C.; Eskeland, S.L.; de Lange, T.; Johansen, D.; Spampinato, C.; Dang-Nguyen, D.-T.; Lux, M.; Schmidt, P.T.; et al. Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection. In Proceedings of the 8th ACM on Multimedia Systems Conference, Taipei, Taiwan, 20–23 June 2017; pp. 164–169. [Google Scholar]
- Gibson, E.; Giganti, F.; Hu, Y.; Bonmati, E.; Bandula, S.; Gurusamy, K.; Davidson, B.; Pereira, S.P.; Clarkson, M.J.; Barratt, D.C. Automatic multi-organ segmentation on abdominal CT with dense V-networks. IEEE Trans. Med. Imaging 2018, 37, 1822–1834. [Google Scholar] [CrossRef] [Green Version]
- Wang, S.; Cong, Y.; Zhu, H.; Chen, X.; Qu, L.; Fan, H.; Zhang, Q.; Liu, M. Multi-scale context-guided deep network for automated lesion segmentation with endoscopy images of the gastrointestinal tract. IEEE J. Biomed. Health Inform. 2020, 25, 514–525. [Google Scholar] [CrossRef]
- Khan, M.A.; Khan, M.A.; Ahmed, F.; Mittal, M.; Goyal, L.M.; Hemanth, D.J.; Satapathy, S.C. Gastrointestinal disease segmentation and classification based on duo-deep architectures. Pattern Recognit. Lett. 2020, 131, 193–204. [Google Scholar] [CrossRef]
- Galdran, A.; Carneiro, G.; Ballester, M.A.G. Double encoder-decoder networks for gastrointestinal polyp segmentation. In International Conference on Pattern Recognition; Springer: Cham, Germany, 2021; pp. 293–307. [Google Scholar]
- Jha, D.; Ali, S.; Emanuelsen, K.; Hicks, S.A.; Thambawita, V.; Garcia-Ceja, E.; Johansen, D.; Halvorsen, P. Kvasir-instrument: Diagnostic and therapeutic tool segmentation dataset in gastrointestinal endoscopy. In International Conference on Multimedia Modeling; Springer: Cham, Germany, 2021; pp. 218–229. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention; Springer: Cham, Germany, 2015; pp. 234–241. [Google Scholar]
- Alokasi, H.; Ahmad, M.B. The Accuracy Performance of Semantic Segmentation Network with Different Backbones. In Proceedings of the 2022 7th International Conference on Data Science and Machine Learning Applications (CDMA), Riyadh, Saudi Arabia, 28 February 2022; pp. 49–54. [Google Scholar]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Gottapu, R.D.; Dagli, C.H. DenseNet for anatomical brain segmentation. Procedia Comput. Sci. 2018, 140, 179–185. [Google Scholar] [CrossRef]
- Siciarz, P.; McCurdy, B. U-net architecture with embedded Inception-ResNet-v2 image encoding modules for automatic segmentation of organs-at-risk in head and neck cancer radiation therapy based on computed tomography scans. Phys. Med. Biol. 2022, 22, 67. [Google Scholar] [CrossRef] [PubMed]
- Le DuyHuynh, N.B. A U-NET++ with pre-trained efficientnet backbone for segmentation of diseases and artifacts in endoscopy images and videos. In Proceedings of the CEUR Workshop Proceedings, Toulouse, France, 7–10 November 2020; Volume 2595, pp. 13–17. [Google Scholar]
- Zhang, Z. Improved adam optimizer for deep neural networks. In Proceedings of the 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS), Banff, AB, Canada, 4–6 June 2018; pp. 1–2. [Google Scholar]
- Agarap, A.F. Deep learning using rectified linear units (relu). arXiv 2018, arXiv:1803.08375. [Google Scholar]
Name of the Block | Name of the Layer | Input Image Size | Filter Size | Number of Filters | Activation Function | Output Image Size | Number of Parameters |
---|---|---|---|---|---|---|---|
Downsampling Block 1 | Input image | 160 × 160 × 1 | ----- | ----- | ----- | 160 × 160 × 1 | ----- |
Conv1 | 160 × 160 × 1 | 3 × 3 | 64 | ReLU | 160 × 160 × 64 | 640 | |
Conv2 | 160 × 160 × 64 | 3 × 3 | 64 | ReLU | 160 × 160 × 64 | 36,928 | |
Downsampling Block 2 | Maxpool | 160 × 160 × 64 | 2 × 2 | 64 | ----- | 80 × 80 × 64 | ----- |
Conv1 | 80 × 80 × 64 | 3 × 3 | 128 | ReLU | 80 × 80 × 128 | 73,856 | |
Conv2 | 80 × 80 × 128 | 3 × 3 | 128 | ReLU | 80 × 80 × 128 | 147,584 | |
Downsampling Block 3 | Maxpool | 80 × 80 × 128 | 2 × 2 | 128 | ----- | 40 × 40 × 128 | ----- |
Conv1 | 40 × 40 × 128 | 3 × 3 | 256 | ReLU | 40 × 40 × 256 | 295,168 | |
Conv2 | 40 × 40 × 256 | 3 × 3 | 256 | ReLU | 40 × 40 × 256 | 590,080 | |
Downsampling Block 4 | Maxpool | 40 × 40 × 256 | 2 × 2 | 256 | ----- | 20 × 20 × 256 | ----- |
Conv1 | 20 × 20 × 256 | 3 × 3 | 512 | ReLU | 20 × 20 × 512 | 1,180,160 | |
Conv2 | 20 × 20 × 512 | 3 × 3 | 512 | ReLU | 20 × 20 × 512 | 2,359,808 | |
Center Block | Maxpool | 20 × 20 × 512 | 2 × 2 | 512 | ----- | 10 × 10 × 512 | ----- |
Conv1 | 10 × 10 × 512 | 3 × 3 | 1024 | ReLU | 10 × 10 × 1024 | 4,719,616 | |
Conv2 | 10 × 10 × 1024 | 3 × 3 | 1024 | ReLU | 10 × 10 × 1024 | 9,438,208 | |
Upsampling Block 1 | Concatenate | 10 × 10 × 1024 | ----- | 1----- | ----- | 20 × 20 × 1024 | ----- |
Conv1 | 20 × 20 × 1024 | 3 × 3 | 512 | ReLU | 20 × 20 × 512 | 9,438,208 | |
Conv2 | 20 × 20 × 512 | 3 × 3 | 512 | ReLU | 20 × 20 × 512 | 2,359,808 | |
Upsampling Block 2 | Concatenate | 20 × 20 × 512 | ----- | ----- | ----- | 40 × 40 × 512 | ----- |
Conv1 | 40 × 40 × 512 | 3 × 3 | 512 | ReLU | 40 × 40 × 512 | 2,359,808 | |
Conv2 | 40 × 40 × 512 | 3 × 3 | 512 | ReLU | 40 × 40 × 512 | 590,080 | |
Upsampling Block 3 | Concatenate | 40 × 40 × 512 | ----- | ----- | ----- | 80 × 80 × 256 | ----- |
Conv1 | 80 × 80 × 256 | 3 × 3 | 128 | ReLU | 80 × 80 × 128 | 590,080 | |
Conv2 | 80 × 80 × 128 | 3 × 3 | 128 | ReLU | 80 × 80 × 128 | 147,584 | |
Upsampling Block 4 | Concatenate | 80 × 80 × 128 | ----- | ----- | ----- | 160 × 160 × 128 | ----- |
Conv1 | 160 × 160 × 128 | 3 × 3 | 64 | ReLU | 160 × 160 × 64 | 147,584 | |
Conv2 | 160 × 160 × 64 | 3 × 3 | 64 | ReLU | 160 × 160 × 64 | 36,928 | |
Conv3 | 160 × 160 × 64 | 3 × 3 | 3 | ReLU | 160 × 160 × 3 | 195 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sharma, N.; Gupta, S.; Koundal, D.; Alyami, S.; Alshahrani, H.; Asiri, Y.; Shaikh, A. U-Net Model with Transfer Learning Model as a Backbone for Segmentation of Gastrointestinal Tract. Bioengineering 2023, 10, 119. https://doi.org/10.3390/bioengineering10010119
Sharma N, Gupta S, Koundal D, Alyami S, Alshahrani H, Asiri Y, Shaikh A. U-Net Model with Transfer Learning Model as a Backbone for Segmentation of Gastrointestinal Tract. Bioengineering. 2023; 10(1):119. https://doi.org/10.3390/bioengineering10010119
Chicago/Turabian StyleSharma, Neha, Sheifali Gupta, Deepika Koundal, Sultan Alyami, Hani Alshahrani, Yousef Asiri, and Asadullah Shaikh. 2023. "U-Net Model with Transfer Learning Model as a Backbone for Segmentation of Gastrointestinal Tract" Bioengineering 10, no. 1: 119. https://doi.org/10.3390/bioengineering10010119
APA StyleSharma, N., Gupta, S., Koundal, D., Alyami, S., Alshahrani, H., Asiri, Y., & Shaikh, A. (2023). U-Net Model with Transfer Learning Model as a Backbone for Segmentation of Gastrointestinal Tract. Bioengineering, 10(1), 119. https://doi.org/10.3390/bioengineering10010119