Deep Multi-Segmentation Approach for the Joint Classification and Segmentation of the Retinal Arterial and Venous Trees in Color Fundus Images ^†

Abstract

The analysis of the retinal vasculature represents a crucial stage in the diagnosis of several diseases. An exhaustive analysis involves segmenting the retinal vessels and classifying them into veins and arteries. In this work, we present an accurate approach, based on deep neural networks, for the joint segmentation and classification of the retinal veins and arteries from color fundus images. The presented approach decomposes this joint task into three related subtasks: the segmentation of arteries, veins and the whole vascular tree. The experiments performed show that our method achieves competitive results in the discrimination of arteries and veins, while clearly enhancing the segmentation of the different structures. Moreover, unlike other approaches, our method allows for the straightforward detection of vessel crossings, and preserves the continuity of the arterial and venous vascular trees at these locations.

1. Introduction

The analysis of the retinal vasculature represents a crucial stage in the diagnosis of several diseases, such as diabetes, age-related macular degeneration (AMD) and glaucoma [1]. This is due to the presence of these diseases causing changes in the retinal vessels. An exhaustive analysis of the retinal vasculature involves segmenting the vascular tree and classifying their vessels into veins and arteries. Despite its utility, this type of analysis is rarely applied in clinical practice, as performing it manually is arduous, and often leads to partly subjective results. For this reason, several automatic methods have been proposed. Early methods addressed these tasks into two sequential steps [2]. However, this approach causes the classification results to be highly conditioned by the segmentation results. To overcome this issue, the current state of the art (SOTA) addresses both tasks as a single multi-class semantic segmentation problem [3,4,5,6].

In this work, we present an accurate approach, based on deep neural networks, for the joint segmentation and classification of the retinal arteries and veins (JSCAV) from color fundus images. This approach, differently to SOTA, decomposes the joint task into three subtasks: the segmentation of arteries, veins and the whole vascular tree. In the following sections, we discuss this approach and its associated advantages.

2. Materials and Methods

The current SOTA formulates the JSCAV task as a single multi-class semantic segmentation problem. However, this approach leads to incomplete segmentation maps for veins and arteries, and does not directly provide vasculature segmentation maps.

As an alternative, we present an approach that decomposes the joint task into three segmentation subtasks [7]. Each of these subtasks addresses the segmentation of one of three classes of interest: arteries, veins and the whole vascular tree. To implement this multi-segmentation (MS) approach, a deep neural network is trained end-to-end using a novel loss function: BCE3. This loss function computes the loss as the sum of the individual segmentation losses of the aforementioned classes. Each individual loss is computed as the binary cross-entropy (BCE) between the predicted probability map and the manually annotated segmentation map. This setting allows for the intuitive handling of vessel crossings, and directly provides precise and complete segmentation maps of the various vascular trees. It also allows for the direct detection of vessel crossings through the element-wise product of the predicted artery and vein maps.

To train and evaluate the networks in the JSCAV task, we employed the publicly available RITE dataset [8], which is composed of 40 color fundus images and their corresponding arteries, veins and vasculature segmentation masks. To facilitate training of the networks, we used the image preprocessing technique specified in [3], as well as online data augmentation. To validate our method, a U-Net network [9] was trained, using both the traditional and the MS approaches.

3. Results and Conclusions

Figure 1 shows an example of an RITE retinography and its arteries, veins, vessels and crossings segmentation maps predicted by a model trained using the MS approach. Figure 2 shows the details of the arteries, veins and vessels segmentation maps of the same retinography predicted by a model trained using the MS and the traditional approaches.

The ablation study performed in the RITE dataset shows that our method provides an adequate performance, especially in the segmentation of the different structures. Notably, the MS approach achieves a mean accuracy of $89.24\pm 0.73$ in the classification of arteries and veins, and an AUC-ROC of $98.33\pm 0.04$ in the segmentation of vessels; for its part, the traditional approach achieves $88.78\pm 0.53$ and $98.07\pm 0.04$, respectively.

In addition, the comparison with the SOTA works in the same dataset, depicted in Figure 3, clearly demonstrates that the presented method achieves competitive results in the discrimination of arteries and veins, while significantly enhancing the vascular segmentation.

Therefore, the presented deep multi-segmentation method allows for the detection of more vessels and to better segment the different structures, while achieving competitive classification results. Furthermore, unlike previous approaches, the method allowsfor the straightforward detection of vessel crossings, as well as preserving the continuity of the arterial and venous vascular trees at these locations (see Figure 2).