UY-NET: A Two-Stage Network to Improve the Result of Detection in Colonoscopy Images
Abstract
:1. Introduction
- UY-Net displays higher spatial accuracy and overall accuracy of polyp detection than other single detection algorithms such as YOLO3-spp, YOLOv4, RetinaNet, and Faster R-CNN. The development and utilization of the two-stage deep learning network, instead of the one-stage or two-stage detection algorithms, can significantly reduce misdiagnosis for colorectal polyps. Patients only receive the most suitable treatment when colorectal polyps are accurately detected. UY-Net can support clinicians in accomplishing this goal.
- For the two-stage network algorithm, the sequence of performing image segmentation followed by image detection assumes a critical role in enhancing its accuracy. Precise segmentation of objects in advance can improve the performance of subsequent detection algorithms. This accounts for why the detection accuracy of UY-Net reaches a significantly high level.
2. Literature Review
- Input can be an inputted image, a patch, or a processed and sampled image;
- Backbone is responsible for pre-training, and a network based on CNNs such as ResNet, CSPDarkNet, AlexNet, DarkNet, or VGGNet is commonly adopted;
- Neck is to extract features at different levels, and another network such as Feature Pyramid Network (FPN), PANet, or Bi-FPN can be chosen to attain this objective;
- Performing U-Net first would result in precise segmentation of abnormalities from the colonoscopy images; after abnormalities were precisely segmented, the subsequent application of YOLOv4 would result in accurate detection of colorectal polyps;
- UY-Net would achieve higher accuracy of polyp detection than the four detectors.
3. Method
3.1. UY-Net
3.2. Dataset
3.3. Intersection over Union (IoU) and Average Precision (AP)
3.4. Settings and Procedures
4. Results and Discussion
- Applying U-Net followed by YOLOv4 results in considerably higher accuracy in detecting colorectal polyps from colonoscopy images.
- 2.
- The two-stage network UY-Net would be more accurate in detecting colorectal polyps than the one-stage or two-stage detection algorithms.
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- 2021 Cause of Death Statistics in Taiwan. Available online: https://www.mohw.gov.tw/cp-16-70314-1.html (accessed on 22 May 2023).
- Overview of Colorectal Cancer Prevention and Control in Taiwan. Available online: https://www.hpa.gov.tw/Pages/Detail.aspx?nodeid=615&pid=1126 (accessed on 22 May 2023).
- Ahn, S.B.; Han, D.S.; Bae, J.H.; Byun, T.J.; Kim, J.P.; Eun, C.S. The miss rate for colorectal adenoma determined by quality-adjusted, back-to-back colonoscopies. Gut Liver 2012, 6, 64–70. [Google Scholar] [CrossRef]
- Heresbach, D.; Barrioz, T.; Lapalus, M.G.; Coumaros, D.; Bauret, P.; Potier, P.; Sautereau, D.; Boustière, C.; Grimaud, J.C.; Barthélémy, C.; et al. Miss rate for colorectal neoplastic polyps: A prospective multicenter study of back-to-back video colonoscopies. Endoscopy 2008, 40, 284–290. [Google Scholar] [CrossRef]
- Vázquez, D.; Bernal, J.; Sánchez, F.J.; Fernández-Esparrach, G.; López, A.M.; Romero, A.; Drozdzal, M.; Courville, A. A benchmark for endoluminal scene segmentation of colonoscopy images. J. Healthc. Eng. 2017, 2017, 4037190. [Google Scholar] [CrossRef]
- de Lange, T.; Halvorsen, P.; Riegler, M. Methodology to develop machine learning algorithms to improve performance in gastrointestinal endoscopy. World J. Gastroenterol. 2018, 24, 5057–5062. [Google Scholar] [CrossRef] [PubMed]
- Hwang, S.; Oh, J.; Tavanapong, W.; Wong, J.; de Groen, P.C. Polyp detection in colonoscopy video using elliptical shape feature. In Proceedings of the 2007 the IEEE International Conference on Image Processing (IEEE-ICIP), San Antonio, TX, USA, 16–19 September 2007; pp. 465–468. [Google Scholar] [CrossRef]
- Ameling, S.; Wirth, S.; Paulus, D.; Lacey, G.; Vilarino, F. Texture-based polyp detection in colonoscopy. In Proceedings of the Bildverarbeitung für die Medizin: Algorithmen-Systeme-Anwendungen, Proceedings of BVM Workshop 2009, Heidelberg, Germany, 22–25 March 2009; Meinzer, H., Deserno, T.M., Handels, H., Tolxdorff, T., Eds.; Springer: Berlin/Heidelberg, Germany, 2009; pp. 346–350. [Google Scholar]
- Karkanis, S.A.; Iakovidis, D.K.; Maroulis, D.E.; Karras, D.A.; Tzivras, M. Computer-aided tumor detection in endoscopic video using color wavelet features. IEEE Trans. Inf. Technol. Biomed. 2003, 7, 141–152. [Google Scholar] [CrossRef] [PubMed]
- Tajbakhsh, N.; Gurudu, S.R.; Liang, J. Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans. Med. Imaging 2016, 35, 630–644. [Google Scholar] [CrossRef] [PubMed]
- Bernal, J.; Sánchez, F.J.; Fernández-Esparrach, G.; Gil, D.; Rodríguez, C.; Vilariño, F. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Comput. Med. Imaging Graph. 2015, 43, 99–111. [Google Scholar] [CrossRef] [PubMed]
- Pozdeev, A.A.; Obukhova, N.A.; Motyko, A.A. Automatic analysis of endoscopic images for polyps detection and segmentation. In Proceedings of the 2019 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (IEEE EIConRus), Saint Petersburg/Moscow, Russia, 28–31 January 2019; pp. 1216–1220. [Google Scholar] [CrossRef]
- Bernal, J.; Tajkbaksh, N.; Sanchez, F.J.; Matuszewski, B.J.; Chen, H.; Yu, L.; Angermann, Q.; Romain, O.; Rustad, B.; Balasingham, I.; et al. Comparative validation of polyp detection methods in video colonoscopy: Results from the MICCAI 2015 endoscopic vision challenge. IEEE Trans. Med. Imaging 2017, 36, 1231–1249. [Google Scholar] [CrossRef]
- Shin, Y.; Qadir, H.A.; Aabakken, L.; Bergsland, J.; Balasingham, I. Automatic colon polyp detection using region based deep CNN and post learning approaches. IEEE Access 2018, 6, 40950–40962. [Google Scholar] [CrossRef]
- Shin, Y.; Qadir, H.A.; Balasingham, I. Abnormal colon polyp image synthesis using conditional adversarial networks for improved detection performance. IEEE Access 2018, 6, 56007–56017. [Google Scholar] [CrossRef]
- Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial nets. In Advances in Neural Information Processing Systems 27, Proceedings of Annual Conference on Neural Information Processing Systems 2014, Montreal, Quebec, Canada, 8–13 December 2014; Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q., Eds.; Curran Associates, Inc.: Boston, MA, USA, 2014; pp. 2672–2680. [Google Scholar]
- Wang, P.; Xiao, X.; Brown, G., Jr.; Berzin, T.M.; Tu, M.; Xiong, F.; Hu, X.; Liu, P.; Song, Y.; Zhang, D.; et al. Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy. Nat. Biomed. Eng. 2018, 2, 741–748. [Google Scholar] [CrossRef] [PubMed]
- Badrinarayanan, V.; Kendall, A.; Cipolla, R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 2481–2495. [Google Scholar] [CrossRef] [PubMed]
- Poorneshwaran, J.M.; Kumar, S.S.; Ram, K.; Joseph, J.; Sivaprakasam, M. Polyp segmentation using generative adversarial network. In Proceedings of the 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin, Germany, 23–27 July 2019; pp. 7201–7204. [Google Scholar] [CrossRef]
- Guo, Y.; Matuszewski, B.J. GIANA polyp segmentation with fully convolutional dilation neural networks. In Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications, Prague, Czech Republic, 25–27 February 2019; pp. 632–641. [Google Scholar]
- Guo, Y.; Bernal, J.; Matuszewski, B.J. Polyp segmentation with fully convolutional deep neural networks-Extended evaluation study. J. Imaging 2020, 6, 69. [Google Scholar] [CrossRef] [PubMed]
- Kang, J.; Gwak, J. Ensemble of instance segmentation models for polyp segmentation in colonoscopy images. IEEE Access 2019, 7, 26440–26447. [Google Scholar] [CrossRef]
- Lee, J.Y.; Jeong, J.; Song, E.M.; Ha, C.; Lee, H.J.; Koo, J.E.; Yang, D.; Kim, N.; Byeon, J. Real-time detection of colon polyps during colonoscopy using deep learning: Systematic validation with four independent datasets. Sci. Rep. 2020, 10, 8379. [Google Scholar] [CrossRef]
- Redmon, J.; Farhadi, A. YOLO9000: Better, faster, stronger. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 6517–6525. [Google Scholar] [CrossRef]
- Lin, T.; Dollár, P.; Girshick, R.; He, K.; Hariharan, B.; Belongie, S. Feature pyramid networks for object detection. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 936–944. [Google Scholar]
- Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1137–1149. [Google Scholar] [CrossRef]
- Bochkovskiy, A.; Wang, C.; Liao, H.M. Yolov4: Optimal speed and accuracy of object detection. arXiv 2020, arXiv:2004.10934. [Google Scholar]
- Redmon, J.; Farhadi, A. YOLOv3: An incremental improvement. arXiv 2018, arXiv:1804.02767. [Google Scholar]
- Wang, C.; Liao, H.M.; Wu, Y.; Chen, P.; Hsieh, J.; Yeh, I. CSPNet: A new backbone that can enhance learning capability of CNN. In Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, 14–19 June 2020; pp. 1571–1580. [Google Scholar] [CrossRef]
- Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 7–12 June 2015; pp. 3431–3440. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015, Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; Navab, N., Hornegger, J., Wells, W., Frangi, A., Eds.; Springer Cham: Basel, Switzerland, 2015; pp. 234–241. [Google Scholar]
- Sharma, N.; Gupta, S.; Koundal, D.; Alyami, S.; Alshahrani, H.; Asiri, Y.; Shaikh, A. U-Net model with transfer learning model as a backbone for segmentation of gastrointestinal tract. Bioengineering 2023, 10, 119. [Google Scholar] [CrossRef]
- Jha, D.; Smedsrud, P.H.; Riegler, M.A.; Halvorsen, P.; de Lange, T.; Johansen, D.; Johansen, H.D. Kvasir-seg: A segmented polyp dataset. In Multimedia Modeling, Proceedings of the 26th International Conference on Multimedia Modeling (MMM 2020), Daejeon, Korea, 5–8 January 2020; Ro, Y.M., Cheng, W., Kim, J., Chu, W., Cui, P., Choi, J., Hu, M., de Neve, W., Eds.; Springer Cham: Basel, Switzerland, 2020; pp. 451–462. [Google Scholar]
- ELKarazle, K.; Raman, V.; Then, P.; Chua, C. Detection of colorectal polyps from colonoscopy using machine learning: A survey on modern techniques. Sensors 2023, 10, 1225. [Google Scholar] [CrossRef]
- Doniyorjon, M.; Madinakhon, R.; Shakhnoza, M.; Cho, Y. An improved method of polyp detection using custom YOLOv4-tiny. Appl. Sci. 2022, 12, 10856. [Google Scholar] [CrossRef]
- de Moura Lima, A.C.; de Paiva, L.F.; Bráz, G., Jr.; de Almeida, J.D.S.; Silva, A.C.; Coimbra, M.T.; de Paiva, A.C. A two–stage method for polyp detection in colonoscopy images based on saliency object extraction and transformers. IEEE Access 2023, 11, 76108–76119. [Google Scholar] [CrossRef]
- Li, J.; Zhang, J.; Chang, D.; Hu, Y. Computer-assisted detection of colonic polyps using improved faster R-CNN. Chin. J. Electron. 2019, 28, 718–724. [Google Scholar] [CrossRef]
- Yang, R.; Yu, Y. Artificial convolutional neural network in object detection and semantic segmentation for medical imaging analysis. Front. Oncol. 2021, 11, 638182. [Google Scholar] [CrossRef]
- Yan, T.; Qin, Y.Y.; Wong, P.K.; Ren, H.; Wong, C.H.; Yao, L.; Hu, Y.; Chan, C.I.; Gao, S.; Chan, P.P. Semantic segmentation of gastric polyps in endoscopic images based on convolutional neural networks and an integrated evaluation approach. Bioengineering 2023, 10, 806. [Google Scholar] [CrossRef]
Algorithm | Learning Rate | Optimizer | Batch Size | Anchors | Loss | Threshold |
---|---|---|---|---|---|---|
Faster R-CNN | 2.5 × 10−4 | Adam | 8 | 256 | L1smooth log loss | 0.4 |
RetinaNet | 1 × 10−5 | SGD | 8 | 15 | L1smooth focal loss | 0.3 |
YOLOv3-spp | 1 × 10−3 | SGD | 16 | 8 | MSE, CE | 0.25 |
YOLOv4 | 1 × 10−3 | SGD | 16 | 8 | CioU, CE | 0.25 |
UY-Net | 1 × 10−3 | SGD | 32 | 18 | CioU, CE | 0.25 |
Algorithm | Backbone | AP | IoU |
---|---|---|---|
Faster R-CNN | ResNet | 0.7866 | 0.5621 |
RetinaNet | ResNet | 0.8697 | 0.7313 |
YOLOv3-spp | ResNet | 0.8105 | 0.8248 |
YOLOv4 | DarkNet | 0.8513 | 0.8205 |
UY-Net | DarkNet, U-Net | 0.9915 | 0.9395 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
He, C.-S.; Wang, C.-J.; Wang, J.-W.; Liu, Y.-C. UY-NET: A Two-Stage Network to Improve the Result of Detection in Colonoscopy Images. Appl. Sci. 2023, 13, 10800. https://doi.org/10.3390/app131910800
He C-S, Wang C-J, Wang J-W, Liu Y-C. UY-NET: A Two-Stage Network to Improve the Result of Detection in Colonoscopy Images. Applied Sciences. 2023; 13(19):10800. https://doi.org/10.3390/app131910800
Chicago/Turabian StyleHe, Cheng-Si, Chen-Ji Wang, Jhong-Wei Wang, and Yuan-Chen Liu. 2023. "UY-NET: A Two-Stage Network to Improve the Result of Detection in Colonoscopy Images" Applied Sciences 13, no. 19: 10800. https://doi.org/10.3390/app131910800
APA StyleHe, C.-S., Wang, C.-J., Wang, J.-W., & Liu, Y.-C. (2023). UY-NET: A Two-Stage Network to Improve the Result of Detection in Colonoscopy Images. Applied Sciences, 13(19), 10800. https://doi.org/10.3390/app131910800