A High-Resolution Network with Strip Attention for Retinal Vessel Segmentation
Abstract
:1. Introduction
2. Related Works
2.1. Deep Convolutional Neural Networks
2.2. Self-Attention Modules
3. Methodology
3.1. Overview of the Network Architecture
3.2. Strip Attention Module
3.3. Loss Function
4. Experiments
4.1. Datasets
4.2. Fundus Image Preprocessing and Patch Extraction
4.3. Implementation Details
4.4. Evaluation Criteria
5. Results and Analysis
5.1. Experiments on DRIVE
5.2. Experiments on STARE
5.3. Cross-Training Experiments
6. Discussion
6.1. Comparison with Other Methods
6.2. Advantages
6.3. Limitations
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Liew, G.; Wang, J.J.; Mitchell, P.; Wong, T.Y. Retinal vascular imaging: A new tool in microvascular disease research. Circ. Cardiovasc. Imaging 2008, 1, 156–161. [Google Scholar] [CrossRef] [PubMed]
- Grogan, A.; Barclay, K.; Colville, D.; Hodgson, L.; Savige, J. Retinal small vessel dilatation in the systemic inflammatory response to surgery. Sci. Rep. 2022, 12, 13291. [Google Scholar] [CrossRef] [PubMed]
- Aleem, S.; Sheng, B.; Li, P.; Yang, P.; Feng, D.D. Fast and accurate retinal identification system: Using retinal blood vasculature landmarks. Trans. Ind. Inform. 2018, 15, 4099–4110. [Google Scholar] [CrossRef]
- Chaudhuri, S.; Chatterjee, S.; Katz, N.; Nelson, M.; Goldbaum, M. Detection of blood vessels in retinal images using two-dimensional matched filters. IEEE Trans. Med. Imaging 1989, 8, 263–269. [Google Scholar] [CrossRef]
- Khandouzi, A.; Ariafar, A.; Mashayekhpour, Z.; Pazira, M.; Baleghi, Y. Retinal vessel segmentation, a review of classic and deep methods. Ann. Biomed. Eng. 2022, 50, 1292–1314. [Google Scholar] [CrossRef] [PubMed]
- Vlachos, M.; Dermatas, E. Multi-scale retinal vessel segmentation using line tracking. Comput. Med. Imaging Graph. 2010, 34, 213–227. [Google Scholar] [CrossRef] [PubMed]
- Li, Q.; You, J.; Zhang, D. Vessel segmentation and width estimation in retinal images using multiscale production of matched filter responses. Expert Syst. Appl. 2012, 39, 7600–7610. [Google Scholar] [CrossRef]
- Wang, S.; Han, Y.; Chen, J.; Pan, Y.; Cao, Y.; Meng, H. Weed classification of remote sensing by UAV in ecological irrigation areas based on deep learning. J. Drain. Irrig. Mach. Eng. 2018, 36, 1137–1141. [Google Scholar]
- Xu, Y.; Wen, D.; Zhou, J.; Fan, X.; Liu, Y. Identification method of cotton seedlings and weeds in Xinjiang based on faster R-CNN. J. Drain. Irrig. Mach. Eng. 2021, 39, 602–607. [Google Scholar]
- Chen, J.; Zhan, Y. Action recognition based on multiple time scale two-stream CNN and confidence fusion in video. J. Jiangsu Univ. Nat. Sci. Ed. 2021, 42, 318–324. [Google Scholar]
- Fan, Y.; Shi, L.; Su, W.; Yan, H. Lane detection algorithm based on PINet + RESA network. J. Jiangsu Univ. Nat. Sci. Ed. 2023, 44, 373–378. [Google Scholar]
- Galdran, A.; Anjos, A.; Dolz, J.; Chakor, H.; Lombaert, H.; Ayed, I.B. State-of-the-art retinal vessel segmentation with minimalistic models. Sci. Rep. 2022, 12, 6174. [Google Scholar] [CrossRef] [PubMed]
- Long, J.; Shelhamer, E.; Darrell, T. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 3431–3440. [Google Scholar]
- Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 5–9 October 2015; Springer: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar]
- Wang, J.; Sun, K.; Cheng, T.; Jiang, B.; Deng, C.; Zhao, Y.; Liu, D.; Mu, Y.; Tan, M.; Wang, X.; et al. Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 43, 3349–3364. [Google Scholar] [CrossRef] [PubMed]
- Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. In Advances in Neural Information Processing Systems; MIT Press: Cambridge, MA, USA, 2017; pp. 5998–6008. [Google Scholar]
- Jin, Q.; Meng, Z.; Pham, T.D.; Chen, Q.; Wei, L.; Su, R. DUNet: A deformable network for retinal vessel segmentation. Knowl. Based Syst. 2019, 178, 149–162. [Google Scholar] [CrossRef]
- Zhang, Y.; Fang, J.; Chen, Y.; Jia, L. Edge-aware U-net with gated convolution for retinal vessel segmentation. Biomed. Signal Process. Control 2022, 73, 103472. [Google Scholar] [CrossRef]
- Yin, P.; Cai, H.; Wu, Q. DF-Net: Deep fusion network for multi-source vessel segmentation. Inf. Fusion 2022, 78, 199–208. [Google Scholar] [CrossRef]
- Li, J.; Gao, G.; Yang, L.; Liu, Y. GDF-Net: A multi-task symmetrical network for retinal vessel segmentation. Biomed. Signal Process. Control 2023, 81, 104426. [Google Scholar] [CrossRef]
- Guo, S. CSGNet: Cascade semantic guided net for retinal vessel segmentation. Biomed. Signal Process. Control 2022, 78, 103930. [Google Scholar] [CrossRef]
- Lin, Z.; Huang, J.; Chen, Y.; Zhang, X.; Zhao, W.; Li, Y.; Lu, L.; Zhan, M.; Jiang, X.; Liang, X. A high resolution representation network with multi-path scale for retinal vessel segmentation. Comput. Methods Programs Biomed. 2021, 208, 106206. [Google Scholar] [CrossRef]
- Guo, M.; Xu, T.; Liu, J.; Liu, Z.; Jiang, P.; Mu, T.; Zhang, S.; Martin, R.R.; Cheng, M.; Hu, S. Attention mechanisms in computer vision: A survey. Comput. Vis. Media 2022, 8, 331–368. [Google Scholar] [CrossRef]
- Corbetta, M.; Shulman, G.L. Control of goal-directed and stimulus-driven attention in the brain. Nat. Rev. Neurosci. 2002, 3, 201–215.1. [Google Scholar] [CrossRef] [PubMed]
- Wang, X.; Girshick, R.; Gupta, A.; He, K. Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18–23 June 2018; pp. 7794–7803. [Google Scholar]
- Fu, J.; Liu, J.; Tian, H.; Li, Y.; Bao, Y.; Fang, Z.; Lu, H. Dual attention network for scene segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 15–20 June 2019; pp. 3146–3154. [Google Scholar]
- Huang, Z.; Wang, X.; Huang, L.; Huang, C.; Wei, Y.; Liu, W. Ccnet: Criss-cross attention for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 603–612. [Google Scholar]
- Li, X.; Zhong, Z.; Wu, J.; Yang, Y.; Lin, Z.; Liu, H. Expectation-maximization attention networks for semantic segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Republic of Korea, 27 October–2 November 2019; pp. 9167–9176. [Google Scholar]
- Li, K.; Qi, X.; Luo, Y.; Yao, Z.; Zhou, X.; Sun, M. Accurate retinal vessel segmentation in color fundus images via fully attention-based networks. IEEE J. Biomed. Health Inform. 2020, 25, 2071–2081. [Google Scholar] [CrossRef] [PubMed]
- Wang, M.; Zhu, W.; Shi, F.; Sun, J.; Chen, H.; Yu, K.; Zhou, Y.; Peng, Y.; Chen, Z.; Chen, X. MsTGANet: Automatic drusen segmentation from retinal OCT images. IEEE Trans. Med. Imaging 2021, 41, 394–406. [Google Scholar] [CrossRef]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26 June–1 July 2016; pp. 770–778. [Google Scholar]
- Milletari, F.; Navab, N.; Ahmadi, S.A. V-net: Fully convolutional neural networks for volumetric medical image segmentation. In Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, 25–28 October 2016; pp. 565–571. [Google Scholar]
- Staal, J.; Abràmoff, M.D.; Niemeijer, M.; Viergever, M.A.; Van Ginneken, B. Ridge-based vessel segmentation in color images of the retina. IEEE Trans. Med. Imaging 2004, 23, 501–509. [Google Scholar] [CrossRef] [PubMed]
- Hoover, A.D.; Kouznetsova, V.; Goldbaum, M. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imaging 2000, 19, 203–210. [Google Scholar] [CrossRef] [PubMed]
- Pizer, S.M.; Amburn, E.P.; Austin, J.D.; Cromartie, R.; Geselowitz, A.; Greer, T.; ter Haar Romeny, B.; Zimmerman, J.B.; Zuiderveld, K. Adaptive histogram equalization and its variations. Comput. Vis. Graph. Image Process. 1987, 39, 355–368. [Google Scholar] [CrossRef]
- Fathi, A.; Naghsh-Nilchi, A.R. Automatic wavelet-based retinal blood vessels segmentation and vessel diameter estimation. Biomed. Signal Process. Control. 2013, 8, 71–80. [Google Scholar] [CrossRef]
- Zhao, Y.; Zhao, J.; Yang, J.; Liu, Y.; Zhao, Y.; Zheng, Y.; Xia, L.; Wang, Y. Saliency driven vasculature segmentation with infinite perimeter active contour model. Neurocomputing 2017, 259, 201–209. [Google Scholar] [CrossRef]
- Li, X.; Jiang, Y.; Li, M.; Yin, S. Lightweight attention convolutional neural network for retinal vessel image segmentation. IEEE Trans. Ind. Inform. 2020, 17, 1958–1967. [Google Scholar] [CrossRef]
- Guo, S. Fundus image segmentation via hierarchical feature learning. Comput. Biol. Med. 2021, 138, 104928. [Google Scholar] [CrossRef]
- Wang, B.; Wang, S.; Qiu, S.; Wei, W.; Wang, H.; He, H. CSU-Net: A context spatial U-Net for accurate blood vessel segmentation in fundus images. IEEE J. Biomed. Health Inform. 2020, 25, 1128–1138. [Google Scholar] [CrossRef] [PubMed]
- Ye, Y.; Pan, C.; Wu, Y.; Wang, S.; Xia, Y. MFI-Net: Multiscale feature interaction network for retinal vessel segmentation. IEEE J. Biomed. Health Inform. 2022, 26, 4551–4562.1. [Google Scholar] [CrossRef] [PubMed]
- Yang, X.; Li, Z.; Guo, Y.; Zhou, D. DCU-net: A deformable convolutional neural network based on cascade U-net for retinal vessel segmentation. Multimed. Tools Appl. 2022, 81, 15593–15607. [Google Scholar] [CrossRef]
- Li, J.; Gao, G.; Liu, Y.; Yang, L. MAGF-Net: A multiscale attention-guided fusion network for retinal vessel segmentation. Measurement 2023, 206, 112316. [Google Scholar] [CrossRef]
- Alvarado-Carrillo, D.E.; Dalmau-Cedeño, O.S. Width attention based convolutional neural network for retinal vessel segmentation. Expert Syst. Appl. 2022, 209, 118313. [Google Scholar] [CrossRef]
- Zhang, Y.; He, M.; Chen, Z.; Hu, K.; Li, X.; Gao, X. Bridge-Net: Context-involved U-net with patch-based loss weight mapping for retinal blood Vessel segmentation. Expert Syst. Appl. 2022, 195, 116526. [Google Scholar] [CrossRef]
Dataset | Quantity | Train-Test Split | Resolution | Format | FoV Mask |
---|---|---|---|---|---|
DRIVE | 40 | 20–20 | .tiff | √ | |
STARE | 20 | leave-one-out | .ppm | × |
Method | ACC (%) | SE (%) | SP (%) | F1 (%) | AUC (%) |
---|---|---|---|---|---|
U-Net | 95.31 | 75.37 | 98.20 | 81.42 | 97.55 |
Baseline | 95.75 | 82.16 | 98.16 | 81.36 | 98.21 |
Proposed | 96.16 | 82.68 | 98.47 | 84.27 | 98.64 |
Method | ACC (%) | SE (%) | SP (%) | F1 (%) | AUC (%) |
---|---|---|---|---|---|
U-Net | 96.28 | 73.27 | 98.67 | 80.89 | 97.64 |
Baseline | 96.64 | 84.62 | 98.48 | 84.23 | 98.42 |
Proposed | 97.08 | 89.36 | 98.47 | 87.85 | 99.21 |
Dataset | Method | ACC (%) | SE (%) | SP (%) | AUC (%) |
---|---|---|---|---|---|
DRIVE (trained on STARE) | Jin et al. [17] | 94.81 | 65.05 | 99.14 | 97.18 |
Li et al. [20] | 95.93 | 72.89 | 99.02 | 97.15 | |
Fathi et al. [36] | 95.81 | 77.68 | 97.59 | 95.16 | |
Zhao et al. [37] | 95.70 | 78.20 | 97.90 | 88.60 | |
Proposed | 95.68 | 81.47 | 98.06 | 96.17 | |
STARE (trained on DRIVE) | Jin et al. [17] | 94.74 | 70.00 | 97.59 | 95.71 |
Li et al. [20] | 95.88 | 70.89 | 97.95 | 96.85 | |
Fathi et al. [36] | 95.91 | 80.61 | 97.17 | 96.80 | |
Zhao et al. [37] | 95.60 | 78.90 | 97.80 | 88.50 | |
Proposed | 96.23 | 84.52 | 98.17 | 97.92 |
Method | Year | ACC (%) | SE (%) | SP (%) | F1 (%) | AUC (%) |
---|---|---|---|---|---|---|
GDF-Net [20] | 2023 | 96.22 | 82.91 | 98.52 | 83.02 | 98.59 |
CSGNet [21] | 2022 | 95.76 | 79.43 | 98.14 | 83.10 | 98.23 |
MPS-Net [22] | 2021 | 95.63 | 83.61 | 97.40 | 82.87 | 98.05 |
Li et al. [38] | 2021 | 95.68 | 79.21 | 98.10 | - | 98.06 |
HHNet [39] | 2021 | 95.75 | 79.93 | 98.06 | 83.06 | 98.22 |
CSU-Net [40] | 2021 | 95.65 | 80.70 | 97.82 | 82.51 | 98.01 |
MFI-Net [41] | 2022 | 95.81 | 81.70 | 97.90 | 83.15 | 98.36 |
DCU-Net [42] | 2022 | 95.68 | 81.15 | 97.80 | 82.72 | 98.10 |
MAGF-Net [43] | 2023 | 95.78 | 82.62 | 97.83 | 83.07 | 98.19 |
Proposed | 2023 | 96.16 | 82.68 | 98.47 | 84.27 | 98.64 |
Method | Year | ACC (%) | SE (%) | SP (%) | F1 (%) | AUC (%) |
---|---|---|---|---|---|---|
GDF-Net [20] | 2023 | 96.53 | 76.16 | 99.57 | 80.22 | 98.89 |
CSGNet [21] | 2022 | 97.10 | 83.57 | 98.62 | 85.42 | 99.10 |
MPS-Net [22] | 2021 | 96.89 | 85.66 | 98.19 | 84.91 | 98.73 |
Li et al. [38] | 2021 | 96.78 | 83.52 | 98.23 | 98.75 | |
CSU-Net [40] | 2021 | 97.02 | 84.32 | 98.45 | 85.16 | 98.25 |
MFI-Net [41] | 2022 | 96.87 | 82.20 | 98.54 | 83.96 | 98.97 |
MAGF-Net [43] | 2023 | 96.49 | 80.93 | 98.44 | 83.64 | 98.98 |
WA-Net [44] | 2022 | 96.65 | 77.67 | 98.77 | 81.76 | 98.65 |
Bridge-Net [45] | 2022 | 96.68 | 80.02 | 98.64 | 82.89 | 99.01 |
Proposed | 2023 | 97.08 | 89.36 | 98.47 | 87.85 | 99.21 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Ye, Z.; Liu, Y.; Jing, T.; He, Z.; Zhou, L. A High-Resolution Network with Strip Attention for Retinal Vessel Segmentation. Sensors 2023, 23, 8899. https://doi.org/10.3390/s23218899
Ye Z, Liu Y, Jing T, He Z, Zhou L. A High-Resolution Network with Strip Attention for Retinal Vessel Segmentation. Sensors. 2023; 23(21):8899. https://doi.org/10.3390/s23218899
Chicago/Turabian StyleYe, Zhipin, Yingqian Liu, Teng Jing, Zhaoming He, and Ling Zhou. 2023. "A High-Resolution Network with Strip Attention for Retinal Vessel Segmentation" Sensors 23, no. 21: 8899. https://doi.org/10.3390/s23218899
APA StyleYe, Z., Liu, Y., Jing, T., He, Z., & Zhou, L. (2023). A High-Resolution Network with Strip Attention for Retinal Vessel Segmentation. Sensors, 23(21), 8899. https://doi.org/10.3390/s23218899