MS-CANet: Multi-Scale Subtraction Network with Coordinate Attention for Retinal Vessel Segmentation

Abstract

Retinal vessel segmentation is crucial in the diagnosis of certain ophthalmic and cardiovascular diseases. Although U-shaped networks have been widely used for retinal vessel segmentation, most of the improved methods have insufficient feature extraction capability and fuse different network layers using element or dimension summation, leading to redundant information and inaccurate retinal vessel localization with blurred vessel edges. The asymmetry of small blood vessels in fundus images also increases the difficulty of segmenting blood vessels. To overcome these challenges, we propose a novel multi-scale subtraction network (MS-CANet) with residual coordinate attention to segment the vessels in retinal vessel images. Our approach incorporates a residual coordinate attention module during the encoding phase, which captures long-range spatial dependencies while preserving precise position information. To obtain rich multi-scale information, we also include multi-scale subtraction units at different perceptual field levels. Moreover, we introduce a parallel channel attention module that enhances the contrast between vessel and background, thereby improving the detection of marginal vessels during the decoding phase. We validate our proposed model on three benchmark datasets, namely DRIVE, CHASE, and STARE. The results demonstrate that our method outperforms most advanced methods under different evaluation metrics.

1. Introduction

Retinal vessels are an essential observable structure in fundus images, and their width, tortuosity, direction, and branching ratio variations are crucial means of diagnosing various diseases [1]. For example, diabetic retinopathy is a microvascular complication caused by elevated blood glucose levels, which can cause retinal vessel swelling [2]. Therefore, retinal vessel segmentation plays a critical role in assisting medical diagnosis. With the ever-increasing number of medical images, manual segmentation by doctors has become time-consuming and laborious and is no longer suitable for modern medical disease screening and diagnosis [3]. Therefore, automatic image segmentation has become a hot research topic in medical image processing. Traditional automatic segmentation methods [4,5,6] cannot accurately segment objects with low contrast and small sizes.

In recent years, machine learning-based methods have rapidly developed and sur-passed traditional methods. Zhao et al. [7] combined the advantages of region-based HMRF and FCM clustering segmentation and proposed a fuzzy clustering image seg-mentation algorithm on the basis of a hidden Markov random field model and Voronoi surface subdivision. Houcemeddine et al. [8] explored the combination of local and global search algorithms in image segmentation by optimizing the MRF model using a mixed ACO-ICM algorithm. Vargas-Muñoz et al. [9] proposed that iterative forest generation (ISF) can generate improved sets of connected superpixels (supervoxels in 3D) through a series of image forest transformations. There are also some image segmentation methods [10,11,12] that have achieved good segmentation results.

With the continual development of deep learning technology, excellent performance has been achieved in the field of image segmentation. In particular, U-Net [13] extracts rich semantic information from the image with the encoder and achieves accurate segmentation with the decoder. To enhance the segmentation performance of U-Net, Zhang and Chung et al. [14] proposed a multi-label architecture that utilized a U-shaped network with residual connections and side outputs to capture multi-scale features. He et al. [15] incorporated local de-regression to obtain additional labels, further improving segmentation accuracy. In addition to the single-branch segmentation methods, dual-branch segmentation methods have also been explored, combining coarse and fine segmentation for better performance. For example, Wu et al. [16] proposed the MS-NFN network, which employs two branches with the same U-shaped structure to improve capillary segmentation. Wang et al. [17] introduced the CTF-Net, a two-branch neural network that generates a preliminary prediction map in one branch and enhances feature extraction capabilities in the other with a feature enhancement module. Multi-scale network frameworks have also gained popularity in recent years. Vessel-Net [18], a multi-scale network proposed by Wu et al., uses four supervised paths to extract semantic features under different scales. Feng et al. [19] introduced CcNet, a cross-connected convolutional neural network that also employs a multi-scale approach for image segmentation. RK A et al. [20] utilizes multi-level loss to integrate features from multiple scales and prediction maps from sub-networks, which improves the ability of the CNN to correlate features from different sensory fields. Other approaches that have been explored include multiscale pooling combined with channel and time attention, which was proposed by W Yu et al. [21]. L Wang et al. [22] proposed the MSAN, a multi-scale attention network that cascades multiple multi-scale attention blocks to balance model complexity and performance efficiently. Although CNNs combined with generative adversarial networks have achieved good results [23], the majority of current methods are still based on U-shaped networks.

At present, although U-shaped-network-based retinal vessel segmentation methods have yielded promising results, they still face several challenges: (1) their feature extraction capability needs improvement; (2) the segmentation of the tiny blood vessel endings continues to be a significant challenge in fundus images due to the asymmetry of capillaries; and (3) most networks use element or dimension summation to merge information among different levels during up-sampling, resulting in redundant information and reduced complementarity of information among different levels.

To address the aforementioned challenges, we propose a novel approach that incorporates a multi-scale subtraction unit to enhance the complementarity of feature maps among different layers by reducing their differences. Additionally, we introduce a residual coordinate attention module and a parallel channel attention module to improve the network’s feature extraction and image up-sampling capabilities. Our proposed method offers several contributions:

• We propose a new multi-scale subtraction network for fundus retinal image segmentation. By deploying multi-scale subtraction units, complementary information from low-order to high-order among layers can be selectively obtained, thereby comprehensively enhancing the perception of vessels regions.
• We add a residual coordinate attention module in the down-sampling stage of the network to improve the ability of the network to extract features. It reduces the loss of feature information due to pooling layers and captures long-range dependencies along one spatial direction while preserving precise location information along another spatial direction.
• We also design a parallel channel attention module and add it to the up-sample output part. It can activate and restore vessels in retinal images, effectively highlighting the boundary information of vessel ends and small vessels.

The remaining sections of this paper are organized as follows. We first describe our proposed network in detail in Section 2. Section 3 introduces the datasets, experimental setting, and evaluation metrics. In Section 4, we discuss and compare experimental results from various aspects. Finally, we draw conclusions in Section 5.

2. Methods

In this section, we provide a comprehensive overview of our proposed network, which includes the composition of the BackBone network, a detailed description of the multi-scale subtraction unit and its deployment, and the design of the residual coordinate attention module and the parallel channel attention module.

2.1. The Network Structure of MS-CANet

Our proposed network, MS-CANet, as shown in Figure 1, consisted of three parts: the encoder part, the multi-scale subtraction part, and the decoder part. The encoder extracted semantic features at different levels of the image using 48 pixels of the original image as input. To better extract image features, we incorporated the Residual Coordinate Attention Module (RCA), which preserved the precise position information of the image and enhanced the representation of blood vessels. Four encoders extracted semantic information at different levels of the image, providing multiple inputs for the next part. The multi-scale subtraction part efficiently obtained complementary information among different levels through multi-level and multi-stage cascaded subtraction operations, enhancing the perception of the vessel region. In the decoder part, feature maps were progressively recovered by up-sampling, and each up-sampling step was fused with different levels of information from the multi-scale subtraction part. Finally, the Parallel Channel Attention Module (PCA) activated and restored the blood vessels in the retinal image, producing the segmentation results in an end-to-end manner.

Figure 1. The structure of MS-CANet.

2.2. Residual Coordinate Attention Module

Location information is crucial for capturing structural features in vision tasks. However, conventional channel attention methods often use global pooling to encode spatial information, which compresses the global spatial information into the channel description, leading to the loss of location information [24]. In order to overcome this limitation, we designed the Residual Coordinate Attention (RCA) module shown in Figure 2. The RCA module aimed to encode channel relationships and long-range dependencies with precise location information, thereby enhancing the representation of vessel features in the network. The use of RCA encouraged the attention module to capture long-range interrelationships in space while retaining accurate location information.

Figure 2. The structure of the residual coordinate attention module.

Given an input $X$, two continuous $3\times 3$ convolutions were used to encode and introduce residual connections to accelerate the learning of the network to obtain a feature map $X_i$. Two spatially scoped pooling kernels were used to average pooling operations for each channel along horizontal and vertical coordinates, respectively, and such operations aggregated features along two spatial directions to obtain a pair of direction-aware feature maps $z_c^h\left(h\right)$ and $z_c^w\left(w\right)$. This pair of feature maps was concatenated in the spatial dimension and encoded with a shared 1 $\times$ 1 convolution function $F_1$ to obtain a pair of intermediate feature maps f with horizontal and vertical spatial information. Then, f was split into two independent tensors $f^h$ and $f^w$ along the spatial dimension, and the $1\times 1$ convolution functions $F_h$ and $F_w$ indifferent spatial directions were then used to transform these two tensors into an input $X_i$ that had the same number of channel tensors. Finally, these two tensors were applied to the original input $X_i$ to obtain the output $Y$. The procedure for calculating the residual coordinate attention module is as follows in Equations (1)–(3):

$$f=\delta \left(F_1\left[z_c^h\left(h\right), z_c^w\left(w\right)\right]\right)$$

(1)

$$g^{\left(h,w\right)}=\delta (F_{\left(h,w\right)}\left(f^{\left(h,w\right)}\right)$$

(2)

$$Y=X_i \times g^h \times g^w$$

(3)

where $\left(h,w\right)$ denotes $h$ or $w$, $\left[z_c^h\left(h\right), z_c^w\left(w\right)\right]$ denotes the tandem operation in spatial dimensions, $\delta$ denotes the Sigmoid activation function, $F_1$ denotes the shared $1\times 1$ convolutional transform function, and $F_h$ and $F_w$ denote the $1\times 1$ convolutional transform functions in the vertical and horizontal spatial directions, respectively.

2.3. Multi-Scale Subtraction Units

To achieve better integration of complementary information from feature layers across multiple levels, we enhanced the original multi-scale subtraction unit [25], which captures only the complementary information between adjacent layers. Although the redundancy of information caused by addition operations was reduced, there still may have been some information redundancy. Therefore, we extended the adjacent levels to each level, enabling high-level features to obtain the complementary information of the lowest-level features. This allowed us to compute across multiple levels and obtain features with multiple receptive fields and different levels of complementarity, which provided richer information for the decoder. The multi-scale subtraction part is depicted in Figure 1, where each subtraction unit performed element-wise subtraction with features at different layers within the same level, ensuring that each layer had complementary information from the other layers.

Let F_a and F_b denote the feature maps activated by the RELU activation function in different layers, and the definition of the multi-scale subtraction unit is as follows in Equation (4):

$$su\_unit=Conv\left(\left|F_a-F_b\right|\right)$$

(4)

2.4. Parallel Channel Attention Module

To activate and enhance the recovery of retinal vessels, we designed a parallel channel attention module at the end of the network, as shown in Figure 3. The input feature map $x$ of size $C\times H\times W$ was sent to the module and processed with $3\times 3$ and $5\times 5$ convolution kernels to generate feature maps $U1$ and $U2$, respectively, under different receptive fields. The same operation was performed on the parallel branches: the global information on each channel of the feature map was obtained through the global average pooling and the sigmoid activation function, and the attention weight coefficients $\alpha$ and $\beta$ of each channel were generated. The attention weight coefficient was applied to the original input feature map to obtain the channel-weighted attention map. After completing the channel weighting of the feature map, the attention maps obtained by the two branches were added element by element to obtain the output of the final module. The calculation formula of the parallel channel attention module is defined as follows in Equation (5):

$$y=U1 · \alpha +U2 · \beta$$

(5)

where $U1$ and $U2$ are the feature maps generated by convolution kernels of 3 and 5, respectively, and $\alpha$ and $\beta$ are the attention weight coefficients.

Figure 3. The structure of the residual coordinate attention module.

3. Datasets and Evaluation Metrics

3.1. Datasets

Our method was validated on three retinal vessel standard datasets, including DRIVE [26], CHASE [27], and STARE [28]. The DRIVE dataset consists of 40 retinal images, 7 of which show signs of mild early diabetic retinopathy. The size of each image is $565\times 584$ pixels, and it was divided 50/50 into training set and test set. The CHASE dataset is a lightweight image dataset for retinal vessel segmentation. It contains 28 retinal images, and each image corresponds to 2 retinal vessel segmentation images. The size of each image is $999\times 960$ pixels. Twenty images were used for training, and the remaining images were used for testing and evaluation.

The STARE dataset consists of 20 images. The size of each image is $700\times 605$ pixels. We used the leave-one-out method for training and testing on this dataset.

3.2. Implementation Details

Our experiment used the deep learning framework PyTorch, the operating system was the Linux operating system, the server used for training was Quardro RTX 6000, and the memory size of the GPU was 24G. During training, random slices were used as the input of the network, and the size of the slices was $48\times 48$ pixels. To expand the data, we rotated and panned each image and expanded the number of slices per image to 10,480. The iteration period of the model training was 100, the batch size was 256, and the initial learning rate was set to 0.001. For the regulation of the learning rate, we used a stepwise decay method, where the decay factor was 0.01, and the weight decay was 0.0005. Using the Adam optimizer, the momentum size was 10⁻⁸. We used the cross-entropy loss function to make the training better.

3.3. Evaluation Metrics

To evaluate the performance of the proposed method, we used a confusion matrix to calculate evaluation indicators such as Sensitivity, Specificity, Accuracy, and F-measure. Each evaluation metric is defined by the following:

$$Accuracy=\frac{TP+TN}{TP+FN+TN+FP}$$

(6)

$$Sensitivity=\frac{TP}{TP+FN}$$

(7)

$$Specificity=\frac{TN}{TN+FP}$$

(8)

$$Precision=\frac{TP}{TP+FP}$$

(9)

$$F\text{-}measure=\frac{2\times Precision\times Sensitivity}{Precision+Sensitivity}$$

(10)

where $TP$ is the number of correctly marked vessels pixels, $TN$ is the number of correctly marked background pixels, $FP$ is the number of background pixels that were incorrectly marked as vessels pixels, and $FN$ is the number of vessels pixels that were incorrectly marked as background pixels.

4. Experiment and Result Analysis

4.1. Structure Ablation

To verify the effectiveness of our proposed residual coordinate attention module (RCA) and parallel channel attention module (PCA), we conducted relevant experiments on the DRIVE and CHASE datasets. Table 1 and Table 2 show the structure ablation experiments on the DRIVE and CHASE datasets, respectively. The BackBone network is a network with a multi-scale subtraction part. On this basis, BackBone/RCA means adding the RCA module to the BackBone. BackBone/PCA means adding the PCA module to the BackBone. BackBone/RCA/PCA means adding both RCA and PCA modules.

Table 1. Comparisons of structure ablation experiments on the DRIVE dataset.

Method	Accuracy	Sensitivity	Specificity	F-Measure
BackBone	0.9701	0.8184	0.9874	0.8288
BackBone/RCA	0.9706	0.8479	0.9846	0.8352
BackBone/PCA	0.9710	0.8365	0.9864	0.8355
BackBone/RCA/PCA	0.9709	0.8432	0.9832	0.8389

Table 2. Comparisons of structure ablation experiments on the CHASE dataset.

Method	Accuracy	Sensitivity	Specificity	F-Measure
BackBone	0.9768	0.8003	0.9877	0.8015
BackBone/RCA	0.9772	0.8148	0.9885	0.8132
BackBone/PCA	0.9769	0.8163	0.9888	0.8128
BackBone/RCA/PCA	0.9782	0.8306	0.9866	0.8171

Table 1 shows that the model with the addition of the RCA module and the PCA module had higher accuracy, sensitivity, and F-measure than the backbone model. It can be seen that the introduction of these two modules allowed the network to have a better learning capability. The RCA module captured the long-range spatial dependencies with precise location information when the network down-sampled to extract features, and it enhanced the expression ability of vessel features in the network. The PCA module increased the disparity between vessels and background, and it had better activation in the decoding stage and recovery of marginal vessels.

As can be seen from Table 2, the networks with the RCA module and the PCA module added, respectively, improved in each performance metric compared with the BackBone, especially the F1-measure, which improved by 1.17% and 1.13%, respectively. Adding both modules to the BackBone (i.e., BackBone/RCA/PCA) resulted in a larger improvement in each performance metric compared with the BackBone. The experiments show that although the accuracy and specificity of BackBone/RCA/PCA were slightly lower than BackBone/RCA, the sensitivity, F1-measure, and ROC were the highest among the four models.

To verify the effectiveness of our added modules, we compared the ROC and PR curves of these four models on the DRIVE and CHASE datasets. Table 3 shows the comparison results. The ROC and PR of the network with the addition of the RCA module and the PCA module were improved. This is due to the addition of coordinate attention and parallel channel attention, better extraction of location information during feature extraction, and better activation and restoration of vessels in the up-sampling stage. This demonstrates the effectiveness of these two modules. After adding both the RCA module and the PCA module to the network, the model (BackBone/RCA/PCA) had the best performance; the ROC and PR were 0.9903 and 0.9441, respectively, which were 0.17% and 0.68% higher than the BackBone network, respectively, also proving that after the two modules were combined, the performance of the BackBone network was improved. On the CHASE dataset, the ROC and PR of the best model (BackBone/RCA/PCA) reached 0.9897 and 0.8977, respectively. Relative to the backbone network, ROC and PR increased by 0.23% and 0.85%, respectively. With the addition of the RCA module and the PCA module, the performance of the model was improved relative to the backbone network, and finally, the model with the two modules added achieved the best performance.

Table 3. Comparisons of ROC and PR curves for structure ablation experiments on DRIVE and CHASE datasets.

Method	DRIVE		CHASE
	ROC	PR	ROC	PR
BackBone	0.9886	0.9373	0.9874	0.8892
BackBone/RCA	0.9885	0.9383	0.9880	0.8927
BackBone/PCA	0.9891	0.9405	0.9876	0.8895
BackBone/RCA/PCA	0.9903	0.9441	0.9897	0.8977

To more intuitively feel the effectiveness of the module in the ablation experiment, we show the visualization results of vessel segmentation on fundus retinal images on DRIVE and CHASE datasets in Figure 4 and Figure 5. From the segmentation results, it can be seen that for different models, it was easy to segment the thicker veins and the slightly thinner arteries in the center of the retina. It was difficult to accurately segment the surrounding tiny vessels; the model needs to have stronger learning ability and feature extraction ability, which is also an important evaluation point for evaluating the effectiveness of segmentation. To highlight the segmentation effect, we show the segmentation results of tiny vessels with red boxes. It can be seen from the segmentation results that the BackBone network could segment the trunk part of the vessels well, but the segmentation effect of the tiny vessels was poor. The network with RCA and PCA modules added, respectively, had a certain degree of improvement in the segmentation of tiny vessels, but there were still some problems, such as edge blurring. Finally, the network with the two modules added could better segment the tiny vessels and contained less noise, which indicates that our method was effective for the segmentation of tiny vessels.

Figure 4. Visualization of ablation experiment on the DRIVE dataset. (a) Original image, (b) Ground Truth, (c) BackBone, (d) BackBone/RCA, (e) BackBone/PCA, and (f) BackBone/RCA/PCA.

Figure 5. Visualization of ablation experiment on the CHASE dataset. (a) Original image, (b) Ground Truth, (c) BackBone, (d) BackBone/RCA, (e) BackBone/PCA, and (f) BackBone/RCA/PCA.

4.2. Attention Module Ablation

The parallel channel attention module (PCA) is designed to better segment retinal vessels and improve the segmentation ability. To verify the effectiveness of the PCA module and verify its better performance compared with other attention modules, we designed a set of ablation experiments. BackBone/RCA was used as the baseline for ablation experiments, and we changed only the attention module of the last layer of up-sampling to verify the effectiveness of the PCA module proposed in this paper. We chose three classical attention modules and PCA modules for experimental comparison. Among them, the first was the efficient channel attention (ECA) of the ECA-Net [29] network, which is usually used for object detection and instance segmentation tasks. This module adopts a local cross-channel interaction strategy without dimensionality reduction. The second was the Convolutional Block Attention Module [30] (CBAM), which performs global average and global maximum mixed pools simultaneously in space and channel, which can obtain more effective information. The last one was the squeeze-and-excitation (SE) block in SENet [31], which improved the performance of model classification by modeling the interdependencies among feature channels.

Table 4 and Table 5 show the experimental results of adding the ECA module, CBAM module, SE module, and our PCA module to the baseline. The experimental results show that the ECA, CBAM, and SE modules improved the performance of the model to a certain extent, but it can be seen from the F-measure evaluation indicator that the overall segmentation result after adding the PCA module was higher than the other three attention modules. Compared with the three modules, the PCA module performed the best because PCA pays great attention to different receptive fields and fuses the features under different receptive fields to maximize activation and restore the representation of vessels.

Table 4. Experimental results of different attention modules on the DRIVE dataset.

Method	Accuracy	Sensitivity	Specificity	F-Measure	ROC	PR
Baseline/ECA	0.9697	0.8613	0.9821	0.8335	0.9883	0.9367
Baseline/CBAM	0.9690	0.7912	0.9892	0.8194	0.9881	0.9347
Baseline/SE	0.9713	0.8303	0.9873	0.8356	0.9895	0.9418
Baseline/PCA	0.9709	0.8432	0.9832	0.8389	0.9903	0.9441

Table 5. Experimental results of different attention modules on the CHASE dataset.

Method	Accuracy	Sensitivity	Specificity	F-Measure	ROC	PR
Baseline/ECA	0.9749	0.7718	0.9886	0.7953	0.9886	0.8726
Baseline/CBAM	0.9712	0.6076	0.9957	0.7272	0.9885	0.8209
Baseline/SE	0.9751	0.8096	0.9862	0.8038	0.9886	0.8819
Baseline/PCA	0.9782	0.8306	0.9866	0.8171	0.9897	0.8977

We also used ROC and PR curves to evaluate the performance of these different attention modules. The final results show that adding the PCA module at the last layer of the network performed better than adding the other three attention modules.

4.3. Model Parameter Quantity and Computation Time Analysis

We compared the number of model parameters and computation time with the classic U-Net model in the field of medical image segmentation. Our model had a total parameter size of 23.99 MB, and it required 4.70 s, 14.35 s, and 6.11 s to segment a complete image on the three standard datasets, DRIVE, CHASE, and STARE, respectively. It required 4 s for the U-Net model to segment a complete image on the DRIVE dataset, and the resulting F-measure was 0.8147. Our model required 0.7 s more than U-Net to segment an image on the DRIVE dataset, but our F-measure reached 0.8389, an improvement of 2.42% compared with U-Net, which is quite a large segmentation accuracy improvement.

4.4. Comparisons with Other Methods

To demonstrate the effectiveness of our proposed vessel segmentation method, we used some segmentation methods from recent years to conduct experiments on these three datasets and compare them with our method. Evaluation metrics included accuracy, sensitivity, specificity, and F1-measure. Table 6 and Table 7 show the retinal vessel segmentation results on the DRIVE and CHASE datasets. Table 8 shows the retinal vessel segmentation results on the STARE dataset. Finally, in Table 9, we show the results of training and testing on the STARE dataset using the leave-one-out method.

Table 6. Comparison of proposed methods with other methods in the DRIVE database.

Method	Year	F-Measure	Accuracy	Sensitivity	Specificity
UNet++ [32]	2018	0.8302	0.9701	0.8120	0.9861
R2U-Net [33]	2018	0.8171	0.9556	0.7792	0.9813
CSNet [34]	2019	-	0.9632	0.8170	0.9854
Vessel-Net [18]	2019	-	0.9578	0.8038	0.9802
D-Net [35]	2019	0.8246	0.9709	0.7839	0.9890
CTF-Net [17]	2020	0.8241	0.9567	0.7849	0.9813
AA-UNet [36]	2020	0.8216	0.9558	-	-
RCED-Net [37]	2021	-	0.9649	0.8252	0.9787
MRA-UNet [38]	2021	0.8293	0.9698	0.8353	0.9828
MFI-Net [39]	2021	0.8318	0.9708	0.8325	0.9838
PLRS-Net [40]	2022	-	0.9682	0.8269	0.9817
Ours	2022	0.8389	0.9709	0.8432	0.9832

Table 7. Comparison of proposed methods with other methods in the CHASE database.

Method	Year	F-Measure	Accuracy	Sensitivity	Specificity
UNet++ [32]	2018	0.8139	0.9760	0.8184	0.9810
R2U-Net [33]	2018	0.7928	0.9634	0.7756	0.9820
Vessel-Net [18]	2019	-	0.9661	0.8132	0.9814
D-Net [35]	2019	0.8062	0.9721	0.7839	0.9894
D-UNet [41]	2019	0.7883	0.9610	-	-
M2UNet [42]	2019	-	0.9703	-	-
AA-UNet [36]	2020	0.7892	0.9608	-	-
RCED-Net [37]	2021	-	0.9772	0.8440	0.8440
MRA-UNet [38]	2021	0.8127	0.9758	0.8324	0.9854
MFI-Net [39]	2021	0.8150	0.9762	0.8309	0.9860
PLRS-Net [40]	2022	-	0.9731	0.8301	0.9893
Ours	2022	0.8171	0.9782	0.8306	0.9866

Table 8. Comparison of proposed methods with other methods in the STARE database.

Method	Year	F-Measure	Accuracy	Sensitivity	Specificity
UNet++ [32]	2018	0.8393	0.9753	0.8646	0.9843
R2U-Net [33]	2018	0.8475	0.9712	0.8298	0.9862
CSNet [34]	2019	-	0.9752	0.8816	0.9840
D-UNet [41]	2019	0.8143	0.9641	-	-
AA-UNet [36]	2020	0.8142	0.9640	-	-
RCED-Net [37]	2021	-	0.9659	0.8397	0.9792
MRA-UNet [38]	2021	0.8422	0.9763	0.8422	0.9873
MFI-Net [39]	2021	0.8483	0.9766	0.8619	0.9859
PLRS-Net [40]	2022	-	0.9715	0.8635	0.9803
Ours	2022	0.8500	0.9772	0.8645	0.9863

Table 9. Results of the leave-one-out method in the STARE database.

Image	Accuracy	Sensitivity	Specificity	F-Measure	ROC	PR
1	0.9771	0.8234	0.9881	0.8278	0.9883	0.9096
2	0.9799	0.8793	0.9863	0.8401	0.9939	0.9320
3	0.9803	0.8754	0.9870	0.8422	0.9935	0.9308
4	0.9651	0.8169	0.9798	0.8088	0.9869	0.8965
5	0.9745	0.9012	0.9800	0.8315	0.9929	0.9273
6	0.9790	0.9064	0.9854	0.8741	0.9947	0.9483
7	0.9791	0.8452	0.9899	0.8581	0.9935	0.9404
8	0.9843	0.8968	0.9918	0.9001	0.9970	0.9695
9	0.9765	0.8407	0.9884	0.8521	0.9931	0.9365
10	0.9806	0.8761	0.9886	0.8657	0.9951	0.9467
11	0.9816	0.9425	0.9849	0.8882	0.9967	0.9645
12	0.9796	0.8976	0.9876	0.8869	0.9955	0.9607
13	0.9790	0.8948	0.9874	0.8855	0.9950	0.9600
14	0.9776	0.8717	0.9876	0.8707	0.9938	0.9459
15	0.9650	0.8404	0.9791	0.8307	0.9859	0.9136
16	0.9761	0.8701	0.9865	0.8671	0.9934	0.9466
17	0.9850	0.8623	0.9915	0.8537	0.9930	0.9311
18	0.9838	0.8087	0.9917	0.8119	0.9925	0.8990
19	0.9701	0.8129	0.9813	0.7843	0.9875	0.8651
20	0.9713	0.8284	0.9837	0.8219	0.9891	0.9078
Average	0.9772	0.8645	0.9863	0.8500	0.9925	0.9315

As can be seen in Table 6, our method achieved good results in all metrics except for the specificity, which was not as high as D-Net. Due to the non-uniform background illumination of the sample images in the CHASE dataset, it was difficult to distinguish vessels from the wider arterial vessels, which requires a model with strong feature extraction capability. In Table 8, compared with the classical UNet++, our model improved on F-measure by 0.32%, accuracy by 0.22%, sensitivity by 1.22%, and specificity by 0.56%. Our method improved on all metrics. As seen in Table 9, we achieved an F-measure of 85.00%, a 1.07% improvement over UNet++ and a 0.25% improvement over R2U-Net. Our method had the highest F-measure and Accuracy.

4.5. Visual Comparison with Different Methods

We compared MS-CANet with the U-Net and UNet++. Figure 6 and Figure 7 show the visualization results for the DRIVE dataset and the CHASE dataset, respectively. From the segmentation results in Figure 6, it can be seen that the retinal vessels segmented by U-Net had a large number of unsegmented edges relative to the ground truth, unclear edge segmentation, and blurred boundaries, among other problems. It can also be seen in Figure 7 that our proposed method had a clearer and more accurate segmentation of the retinal vessel endings than UNet++.

Figure 6. Visual comparison of segmentation results on the DRIVE dataset. (a) Original Image, (b) Ground Truth, (c) U-Net, and (d) Ours.

Figure 7. Visual comparison of segmentation results on the CHASE dataset. (a) Original Image, (b) Ground Truth, (c) U-Net++, and (d) Ours.

Our proposed method used a residual coordinate attention module for down-sampling, which preserved accurate location information while establishing long-range dependencies, and it better extracted the information of marginal tiny vessels. The parallel channel attention in up-sampling enabled the network to distinguish foreground and background regions well, resulting in more accurate segmentation results and containing less noise, especially in the vessel regions in the red box in the figure.

4.6. Robustness Testing

The acquisition methods for retinal images are different, and the three datasets used in the experiments were obtained using different methods. Therefore, there are often differences in illumination, noise, and other factors in clinical retinal images. To verify the generalization ability of MS-CANet on different datasets, we designed a multi-dataset cross-testing experiment. Cross-testing refers to training the model on one dataset and testing it on another. Table 10 displays the results of cross-testing on multiple datasets, with the vertical header indicating the dataset used for training and the horizontal header indicating the dataset used for testing.

Table 10. Cross-test results.

		F-Measure			Accuracy
	DRIVE	CHASE	STARE	DRIVE	CHASE	STARE
DRIVE	0.8389	0.5482	0.7826	0.9709	0.9541	0.9673
CHASE	0.6849	0.8171	0.7952	0.9648	0.9782	0.9716
STARE	0.8035	0.6521	0.8500	0.9701	0.9648	0.9772

From the results in Table 10, it can be seen that the F-measure of the model trained on the DRIVE dataset decreased by 29.01% and 5.63%, and the accuracy decreased by 1.68% and 0.36% when tested on the CHASE and STARE datasets, respectively. When the model trained on CHASE was tested on the other two data sets, F-measure decreased by 13.22% and 2.19%, and accuracy decreased by 1.34% and 0.66%, respectively. When the model trained on STARE was tested on the other two datasets, the F-measure decreased by 4.65% and 19.79%, and the accuracy decreased by 0.71% and 1.24%, respectively. From the results of the cross-test, the model trained on the DRIVE dataset had a larger drop in metric value during the cross-test, and fewer general features were extracted on the DRIVE dataset because the images are smaller and the differences between images are smaller. The models trained on the CHASE and STARE datasets had strong generalization ability because the CHASE dataset contains more feature information and the differences between images are larger. The overall difference between the STARE dataset images is relatively small, and a large amount of useful feature information can be extracted, but the difference in the lesion area is large.

5. Conclusions and Future Work

Retinal vessel segmentation is of great significance for the diagnosis and prediction of various ophthalmic diseases. We proposed the MS-CANet network to segment images of fundus retinal vessels. We designed a multi-scale subtraction unit to eliminate the redundancy of information between different levels. The long-range dependencies of spatial orientation were captured by adding the residual coordinate attention module while ensuring accurate location information. The parallel channel attention module we designed can effectively promote the recovery of tiny vessels at the edge during the up-sampling process, increase the difference between vessels and the background, and make the network more sensitive to the vessels. Finally, we validated our method on three standard retinal datasets.

Although our method achieved good performance, there are still some problems that need to be solved. We employed supervised learning, which requires a large number of labeled datasets, but large medically labeled datasets are expensive and not easy to obtain. In addition, we validated our network only on the retinal vessel segmentation task, which has some limitations. This is also a focus of our future work, applying our method to multiple medical image segmentation tasks.