A Multi-Scale Liver Tumor Segmentation Method Based on Residual and Hybrid Attention Enhanced Network with Contextual Integration
Abstract
:1. Introduction
2. Related Work
2.1. Multi-Scale Feature Fusion
2.2. Attention Mechanism
3. Methods
3.1. Residual Module
3.2. Hybrid Gated Attention
3.3. Multi-Scale Feature Enhancement
3.4. Loss Function
4. Experiment Results
4.1. Experimental Configuration
4.2. Data
4.3. Data Preprocessing
4.4. Ablation Experiment
4.5. Analysis of Training Loss Rate
4.6. Liver Segmentation Results
4.7. Tumor Segmentation Results
5. Discussion and Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Waller, L.P.; Deshpande, V.; Pyrsopoulos, N. Hepatocellular carcinoma: A comprehensive review. World J. Hepatol. 2015, 7, 2648. [Google Scholar] [CrossRef] [PubMed]
- Bray, F.; Ferlay, J.; Soerjomataram, I.; Siegel, R.L.; Torre, L.A.; Jemal, A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA A Cancer J. Clin. 2018, 68, 394–424. [Google Scholar] [CrossRef]
- Kelley, R.K.; Greten, T.F. Hepatocellular carcinoma—Origins and outcomes. N. Engl. J. Med. 2021, 385, 280–282. [Google Scholar] [CrossRef]
- Bosch, F.X.; Ribes, J.; Díaz, M.; Cléries, R. Primary liver cancer: Worldwide incidence and trends. Gastroenterology 2004, 127, S5–S16. [Google Scholar] [CrossRef]
- Thong, W.; Kadoury, S.; Piché, N.; Pal, C.J. Convolutional networks for kidney segmentation in contrast-enhanced CT scans. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 2018, 6, 277–282. [Google Scholar] [CrossRef]
- Yadav, S.; Patra, S.; Arora, C.; Banerjee, S. Deep CNN with color lines model for unmarked road segmentation. In Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China, 17–20 September 2017; pp. 585–589. [Google Scholar]
- Chartrand, G.; Cresson, T.; Chav, R.; Gotra, A.; Tang, A.; De Guise, J.A. Liver segmentation on CT and MR using Laplacian mesh optimization. IEEE Trans. Biomed. Eng. 2016, 64, 2110–2121. [Google Scholar] [CrossRef]
- Araújo, J.D.L.; da Cruz, L.B.; Diniz, J.O.B.; Ferreira, J.L.; Silva, A.C.; de Paiva, A.C.; Gattass, M. Liver segmentation from computed tomography images using cascade deep learning. Comput. Biol. Med. 2022, 140, 105095. [Google Scholar] [CrossRef]
- Otsu, N. A threshold selection method from gray-level histograms. Automatica 1975, 11, 23–27. [Google Scholar] [CrossRef]
- Das, K.; Bhowmik, M.K.; Chowdhuary, O.; Bhattacharjee, D.; De, B.K. Accurate segmentation of inflammatory and abnormal regions using medical thermal imagery. Australas. Phys. Eng. Sci. Med. 2019, 42, 647–657. [Google Scholar] [CrossRef]
- Bangare, S.L. Classification of optimal brain tissue using dynamic region growing and fuzzy min-max neural network in brain magnetic resonance images. Neurosci. Inform. 2022, 2, 100019. [Google Scholar] [CrossRef]
- Guo, L.; Chen, L.; Chen, C.P.; Zhou, J. Integrating guided filter into fuzzy clustering for noisy image segmentation. Digit. Signal Process. 2018, 83, 235–248. [Google Scholar] [CrossRef]
- Mirghasemi, S.; Andreae, P.; Zhang, M. Domain-independent severely noisy image segmentation via adaptive wavelet shrinkage using particle swarm optimization and fuzzy C-means. Expert Syst. Appl. 2019, 133, 126–150. [Google Scholar] [CrossRef]
- Fen, W.; Li, H. Application of mean shift clustering method in traditional clothing image extraction. Wool Text. J. 2022, 50, 89. [Google Scholar]
- Jeslin Libisha, J; Pavithra, G.; Sundar, R.; Ram Kumar, C. Liver Tumor Detection by Automated Thresholding and Image Segmentation. Res. Sq. 2022. preprint.
- Das, A.; Sabut, S.K. Kernelized fuzzy C-means clustering with adaptive thresholding for segmenting liver tumors. Procedia Comput. Sci. 2016, 92, 389–395. [Google Scholar] [CrossRef]
- Xu, L.; Zhu, Y.; Zhang, Y.; Yang, H. Liver segmentation based on region growing and level set active contour model with new signed pressure force function. Optik 2020, 202, 163705. [Google Scholar] [CrossRef]
- Sridhar, K.; Lai, W.-C.; Kavin, B.P. Detection of liver tumour using deep learning based segmentation with coot extreme learning model. Biomedicines 2023, 11, 800. [Google Scholar] [CrossRef]
- Chen, M.; Liu, W.; Wang, T.; Zhang, S.; Liu, A. A game-based deep reinforcement learning approach for energy-efficient computation in MEC systems. Knowl.-Based Syst. 2022, 235, 107660. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, 5–9 October 2015; proceedings, part III 18. pp. 234–241. [Google Scholar]
- Liu, X.; Yang, L.; Chen, J.; Yu, S.; Li, K. Region-to-boundary deep learning model with multi-scale feature fusion for medical image segmentation. Biomed. Signal Process. Control 2022, 71, 103165. [Google Scholar] [CrossRef]
- Oktay, O.; Schlemper, J.; Folgoc, L.L.; Lee, M.; Heinrich, M.; Misawa, K.; Mori, K.; McDonagh, S.; Hammerla, N.Y.; Kainz, B. Attention u-net: Learning where to look for the pancreas. arXiv 2018, arXiv:1804.03999. [Google Scholar]
- Jha, D.; Smedsrud, P.H.; Johansen, D.; De Lange, T.; Johansen, H.D.; Halvorsen, P.; Riegler, M.A. A comprehensive study on colorectal polyp segmentation with ResUNet++, conditional random field and test-time augmentation. IEEE J. Biomed. Health Inform. 2021, 25, 2029–2040. [Google Scholar] [CrossRef]
- Hu, J.; Shen, L.; Sun, G. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7132–7141. [Google Scholar]
- Chen, L.-C.; Papandreou, G.; Schroff, F.; Adam, H. Rethinking atrous convolution for semantic image segmentation. arXiv 2017, arXiv:1706.05587. [Google Scholar]
- Yu, F.; Koltun, V. Multi-scale context aggregation by dilated convolutions. arXiv 2015, arXiv:1511.07122. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37, 1904–1916. [Google Scholar] [CrossRef] [PubMed]
- Chen, L.-C.; Zhu, Y.; Papandreou, G.; Schroff, F.; Adam, H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the European conference on computer vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 801–818. [Google Scholar]
- Roy, A.G.; Navab, N.; Wachinger, C. Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks. In Proceedings of the Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, 16–20 September 2018; Proceedings, Part I. pp. 421–429. [Google Scholar]
- Woo, S.; Park, J.; Lee, J.-Y.; Kweon, I.S. Cbam: Convolutional block attention module. In Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 3–19. [Google Scholar]
- Wang, Z.; Zou, Y.; Liu, P.X. Hybrid dilation and attention residual U-Net for medical image segmentation. Comput. Biol. Med. 2021, 134, 104449. [Google Scholar] [CrossRef] [PubMed]
- Ates, G.C.; Mohan, P.; Celik, E. Dual cross-attention for medical image segmentation. Eng. Appl. Artif. Intell. 2023, 126, 107139. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 770–778. [Google Scholar]
- Li, H.; Zeng, N.; Wu, P.; Clawson, K. Cov-Net: A computer-aided diagnosis method for recognizing COVID-19 from chest X-ray images via machine vision. Expert Syst. Appl. 2022, 207, 118029. [Google Scholar] [CrossRef] [PubMed]
- Zhang, Z.; Wu, C.; Coleman, S.; Kerr, D. DENSE-INception U-net for medical image segmentation. Comput. Methods Programs Biomed. 2020, 192, 105395. [Google Scholar] [CrossRef]
- Zhang, J.; Xie, Y.; Zhang, P.; Chen, H.; Xia, Y.; Shen, C. Light-Weight Hybrid Convolutional Network for Liver Tumor Segmentation. In Proceedings of the IJCAI, Macao, China, 10–16 August 2019; pp. 4271–4277. [Google Scholar]
- Gu, Z.; Zhou, S.; Niu, L.; Zhao, Z.; Zhang, L. Context-aware feature generation for zero-shot semantic segmentation. In Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA, 12–16 October 2020; pp. 1921–1929. [Google Scholar]
- Bilic, P.; Christ, P.; Li, H.B.; Vorontsov, E.; Ben-Cohen, A.; Kaissis, G.; Szeskin, A.; Jacobs, C.; Mamani, G.E.H.; Chartrand, G. The liver tumor segmentation benchmark (lits). Med. Image Anal. 2023, 84, 102680. [Google Scholar] [CrossRef]
- Fu, Q.; Celenk, M.; Wu, A. An improved algorithm based on CLAHE for ultrasonic well logging image enhancement. Clust. Comput. 2019, 22, 12609–12618. [Google Scholar] [CrossRef]
- Lv, P.; Wang, J.; Wang, H. 2.5 D lightweight RIU-Net for automatic liver and tumor segmentation from CT. Biomed. Signal Process. Control 2022, 75, 103567. [Google Scholar] [CrossRef]
- Diakogiannis, F.I.; Waldner, F.; Caccetta, P.; Wu, C. ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS J. Photogramm. Remote Sens. 2020, 162, 94–114. [Google Scholar] [CrossRef]
- Gu, R.; Wang, G.; Song, T.; Huang, R.; Aertsen, M.; Deprest, J.; Ourselin, S.; Vercauteren, T.; Zhang, S. CA-Net: Comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE Trans. Med. Imaging 2020, 40, 699–711. [Google Scholar] [CrossRef] [PubMed]
- Zeng, N.; Li, X.; Wu, P.; Li, H.; Luo, X. A novel tensor decomposition-based efficient detector for low-altitude aerial objects with knowledge distillation scheme. IEEE/CAA J. Autom. Sin. 2024, 11, 487–501. [Google Scholar] [CrossRef]
- Xie, T.; Wang, Z.; Li, H.; Wu, P.; Huang, H.; Zhang, H.; Alsaadi, F.E.; Zeng, N. Progressive attention integration-based multi-scale efficient network for medical imaging analysis with application to COVID-19 diagnosis. Comput. Biol. Med. 2023, 159, 106947. [Google Scholar] [CrossRef] [PubMed]
- Chen, X.; Wang, X.; Zhang, K.; Fung, K.-M.; Thai, T.C.; Moore, K.; Mannel, R.S.; Liu, H.; Zheng, B.; Qiu, Y. Recent advances and clinical applications of deep learning in medical image analysis. Med. Image Anal. 2022, 79, 102444. [Google Scholar] [CrossRef]
Model | Dice | Jaccard | Precision | Recall |
---|---|---|---|---|
U-Net | 94.27 | 89.51 | 95.17 | 94.96 |
Res+U-Net | 94.85 | 90.49 | 95.74 | 95.65 |
HGA+U-Net | 95.18 | 90.85 | 95.45 | 95.56 |
MSFE+U-Net | 94.79 | 91.23 | 95.23 | 94.97 |
Res+HGA+U-Net | 95.39 | 90.91 | 95.42 | 95.38 |
RHEU-Net(ours) | 95.72 | 91.49 | 96.23 | 95.98 |
Model | Dice | Jaccard | Precision | Recall |
---|---|---|---|---|
U-Net | 66.83 | 56.82 | 82.41 | 65.68 |
Res+U-Net | 68.31 | 58.43 | 83.79 | 68.87 |
HGA+U-Net | 68.66 | 58.25 | 84.83 | 68.95 |
MSFE+U-Net | 68.04 | 58.17 | 83.06 | 68.89 |
Res+HGA+U-Net | 69.47 | 60.12 | 85.24 | 70.62 |
RHEU-Net | 70.19 | 61.26 | 85.45 | 71.79 |
Model | Dice | Jaccard | Precision | Recall |
---|---|---|---|---|
U-Net [20] (2015) | 94.27 | 89.51 | 95.17 | 94.96 |
AttentionUnet [22] (2018) | 95.23 | 90.51 | 95.76 | 96.14 |
ResUnet-a [41] (2020) | 95.13 | 90.24 | 95.65 | 95.94 |
CANet [42] (2020) | 94.86 | 89.52 | 95.86 | 95.42 |
Res Unet++ [23] (2021) | 94.49 | 89.64 | 95.84 | 95.29 |
RIUNet [40] (2022) | 95.38 | 90.98 | 95.89 | 96.23 |
RHEU-Net(ours) | 95.72 | 91.49 | 96.23 | 95.98 |
Model | Dice | Jaccard | Precision | Recall |
---|---|---|---|---|
U-Net [20] (2015) | 66.83 | 56.82 | 82.41 | 65.68 |
AttentionUnet [22] (2018) | 69.65 | 61.43 | 85.31 | 71.63 |
ResUnet-a [41] (2020) | 67.19 | 57.92 | 85.03 | 66.58 |
CANet [42] (2020) | 67.83 | 60.05 | 82.72 | 66.35 |
Res Unet++ [23] (2021) | 67.45 | 59.49 | 82.28 | 65.82 |
RIUNet [40] (2022) | 69.77 | 61.35 | 85.39 | 71.72 |
RHEU-Net(ours) | 70.19 | 61.26 | 85.45 | 71.79 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Sun, L.; Jiang, L.; Wang, M.; Wang, Z.; Xin, Y. A Multi-Scale Liver Tumor Segmentation Method Based on Residual and Hybrid Attention Enhanced Network with Contextual Integration. Sensors 2024, 24, 5845. https://doi.org/10.3390/s24175845
Sun L, Jiang L, Wang M, Wang Z, Xin Y. A Multi-Scale Liver Tumor Segmentation Method Based on Residual and Hybrid Attention Enhanced Network with Contextual Integration. Sensors. 2024; 24(17):5845. https://doi.org/10.3390/s24175845
Chicago/Turabian StyleSun, Liyan, Linqing Jiang, Mingcong Wang, Zhenyan Wang, and Yi Xin. 2024. "A Multi-Scale Liver Tumor Segmentation Method Based on Residual and Hybrid Attention Enhanced Network with Contextual Integration" Sensors 24, no. 17: 5845. https://doi.org/10.3390/s24175845
APA StyleSun, L., Jiang, L., Wang, M., Wang, Z., & Xin, Y. (2024). A Multi-Scale Liver Tumor Segmentation Method Based on Residual and Hybrid Attention Enhanced Network with Contextual Integration. Sensors, 24(17), 5845. https://doi.org/10.3390/s24175845