A Survey on Diabetic Retinopathy Lesion Detection and Segmentation

Abstract

Diabetes is a global problem which impacts people of all ages. Diabetic retinopathy (DR) is a main ailment of the eyes resulting from diabetes which can result in loss of eyesight if not detected and treated on time. The current process of detecting DR and its progress involves manual examination by experts, which is time-consuming. Extracting the retinal vasculature, and segmentation of the optic disc (OD)/fovea play a significant part in detecting DR. Detecting DR lesions like microaneurysms (MA), hemorrhages (HM), and exudates (EX), helps to establish the current stage of DR. Recently with the advancement in artificial intelligence (AI), and deep learning(DL), which is a division of AI, is widely being used in DR related studies. Our study surveys the latest literature in “DR segmentation and lesion detection from fundus images using DL”.

1. Introduction

The number of people diagnosed with diabetes has been increasing at an alarming rate during the past two decades. This accounts for almost half a billion people in the world as per the IDF Diabetes Atlas [1], which makes it a global health concern. This includes people of all ages. By 2045, this figure is supposed to reach seven hundred million and by the year 2040, one in three persons with this ailment will have DR, which is an ailment characterized by the occurrence of damaged blood vessels in the back of the retina. If it is not detected in a timely manner, it can lead to serious problems, like loss of sight. This makes DR a topic of the utmost importance. Doctors must physically analyze fundus images to determine the presence of this disease and to grade it. Fundus images offer graphical records that detail the current ophthalmic form of an indvidual’s retina. This is a very time-consuming process, and in many countries the number of available trained professionals may not be proportionate to the number of patients. This may hinder providing timely treatment to many patients. Diabetes patients are instructed by doctors to have a routine medical screening of their fundus. Still many DR cases are left unnoticed until they reach a critical state. Due to this reason, it is advantageous to have computerized systems that assist in the diagnosis of DR.

Researchers are developing automated systems for DR diagnosis by identifying the existence of certain DR lesions in fundus images. Few of the DR lesions include MA, HM, EX consisting of soft exudates (SE) and hard exudates(HE). DR grades are also established with the help of lesions present in the fundus. For example, the preliminary stage of DR is marked by the beginning of the growth of new blood vessels whose walls weaken, and tiny bulges or MA protrude from them. Later, when DR progresses to proliferative DR, other lesions like HE and SE may develop. HE are little white/yellowish-white deposits with crisp boundaries whereas, SE/Cotton-wool spots are light yellow or white areas which have unclear edges. Additionally, abnormal blood vessels may grow and bleed causing HM. Hence, some researchers are focusing on the detection and segmentation of DR lesions like MA, HM, and EX which will aid in developing better DR diagnoses/grading for automated systems. For example, Figure 1 shows each of these DR lesions, whereas Figure 2 provides a visualization of a healthy retina vs an unhealthy retina.

Figure 1. Diabetic Retinopathy Lesions [2].

Figure 2. Visualization of a healthy retina and an unhealthy retina [3].

There are many challenges to developing such automated systems. The size of these lesions vary in a temporal manner and the OD, and fovea may be mistaken for lesions. Due to this reason, some researchers are focusing on OD/fovea detection which can also help to improve such systems. Another category of studies that are related to DR include retinal blood vessel segmentation studies. These are important to detect several medical disorders like DR and hypertensive retinopathy. Atypical variations in the retinal vessel will give an idea of the severity of several non-ophthalmic ailments like DR, diabetes mellitus, hypertension, arteriosclerosis, cardiovascular disease, and stroke [4]. Hence, it is important for diagnosing these diseases, especially DR.

Several DL methods were used by researchers to accomplish these types of DR studies. We present a review of the current literature in this field by emphasising how DL has been applied to DR segmentation and lesion detection from fundus images. Deep learning is a section of artificial intelligence which uses artificial neural networks with many processing layers for steadily extracting high-level features from the data.

This article is organized as follows: the studies that were reviewed associated with segmentation and lesion detection are shown in Section 2. Section 3 illustrates the datasets used. These are followed by an evaluation and discussion of these methods in Section 4. After this we have compiled a list of some possible future directions in this area of study in Section 5. Lastly, a conclusion is provided in Section 6.

2. Related Works

The DR studies reviewed in this article have been classified into two categories which include traditional-based approaches and DL-based studies.

Different traditional methods are being used for retinal blood vessel segmentation as well as for lesion detection from fundus images. Detection and counting of MA were done by the authors in [5] by using morphological operations, histogram equalization, and intensity transformation functions. HM segmentation was performed by the authors of [6] using histogram equalization and morphological operations on the green channel image of fundus images.

Several traditional methods have been used for EX detection like thresholding, morphology, edge detection, mixture modeling, and region growing. Grayscale morphology was used along with active contour-based segmentation and region-wise classification by the authors of [7] for automatic EX detection. The authors used a blend of region growing and edge detection in [8]. They used the Luv color space to get the color difference image of objects. A mean squared Wiener filter in two dimensions was used to remove noise which was trailed by edge detection and an enhancement of the region growing method. Gray level thresholding was used by the authors in [9] since EX have high local and gray levels. The local properties of EX were exploited and a blend of global as well as local histogram thresholding was used by them. Sanchez et al. [10] used an EX detection method built using mixture models for obtaining image histograms. This helped them get a dynamic threshold for every image. Some machine learning approaches have also been used in the literature. Niemeijer et al. [11] used the k-nearest neighbor classifier for detecting red lesions from fundus images. A ranking SVM-boosted convolutional neural network (CNN) was utilized for EX detection by the authors of [12]. Damaged or falsely detected OD and HM were also detected by them.

Retinal blood vessel segmentation using a top-hat morphological transform followed by hybrid median filtering was performed by the authors in [13]. Quinn and Krishnan [14] performed a contrast adjustment of the green channel of the image trailed by curvelet transform and morphological reconstruction for retinal blood vessel segmentation. A mathematical morphology-based fuzzy algorithm method was used for OD elimination followed by retinal blood vessel extraction by the authors of [15].

The DL-based DR studies reviewed in this study have been categorized into two types which include segmentation-based studies and lesion detection-based studies. Figure 3 captures these two distinct types of DR studies.

Figure 3. Categorisation of Diabetic Retinopathy studies reviewed.

2.1. Diabetic Retinopathy Lesion Detection

The studies that were devoted to the detection of DR lesions like MA, HM, and EX fall under this category. Another group of studies detected multiple lesions as well as OD. A set of proposed methods are discussed in this section as well as a summarization of them in terms of techniques and datasets used in Table 1 summarizes.

Some studies reviewed, focused on detecting EX. Raja and Balaji [16] used adaptive histogram equalization for preprocessing followed by blood vessel segmentation using CNN. After this, EX segmentation was performed using fuzzy c-means clustering. Finally, DR classification was achieved using SVM. Adem [17], combined a circular Hough transform with CNNs to detect EX. The circular Hough transform helps to ignore OD. EX detection was performed after segmentation of EX. SVM based on the ranking was implemented along with CNN architecture to accurately segment fundus images by the authors in [12]. They also detected EX after segmentation. HE detection with deep CNN and multi-feature joint representation was performed by Wang et al. [18] using a new methodology. They utilized combined features that included deep features as well as handcrafted features. HE detection using SVM blended with a speedier region-based CNN (RCNN) was performed by the authors of [19]. Wang et al. [20] performed EX classification. They used a CNN to investigate if the classification performance improves if more importance was given to the image characteristic inside an EX. The outcomes showed that the performance improved. Recurrent neural network (RNN) based semantic features were used for EX segmentation by the authors of [21] for detecting DR. Huang et al. [22] used a super-pixel segmentation algorithm, followed by multi-feature extraction and patch generation. Finally, CNN was used for HE detection. RCNN with MobileNet was utilized as a feature extractor by the authors of [23] for EX detection. Mateen et al. [24] used pre-trained Inception-v3, ResNet-50, and VGG-19 to perform feature extraction before EX classification using the softmax classifier. Another study by the authors of [25] used three different models including SqueezeNet, ResNet50, and GoogLeNet for EX detection. Among these three, ResNet50 was found to provide the best results. A CNN with residual skip connection was used for EX detection by the authors in [26]. The architecture used was called RTC-net.

Few studies focused on MA detection. Multi-scale residual network (MSRNet) was used for segmentation by the authors of [27] to detect tiny MA. It was based on U-Net architecture. After this, classification was achieved using MS-EfficientNet (multi-scale efficient network). Qiao et al. [28] used the prognosis of MA for DR grading into three, which includes early NPDR, moderate NPDR, and severe NPDR. Deep CNN was used to perform semantic segmentation of fundus images. Gaussian answer Laplacian (LOG) and maximum matching filter response (MFR) filters were used for lesion identification. Feature extraction using VGG and Inception V3 backbones, alongside transfer learning was utilized for MA detection by the authors of [29]. After this, several machine learning classifiers were used for classifying input images. Qomariah et al. [30] modified the U-Net model and used it for MA segmentation. Residual units with modified identity mapping were used in this new model known as MResUNet. Subhasree et al. [31] made use of three different transfer learning-based CNN models for MA detection. These consisted of VGG-16, Inception V3, and ResNet50. They also performed the same task by making use of a random forest approach.

Some studies did a general detection of DR lesions. Conditional generative adversarial network (cGAN) was used for pixel-level lesion segmentation by combining with holistically-nested edge detection (HEDNet) by the authors in [32], and it was found that adversarial loss advances the lesion segmentation performance. An approach known as super-pixel-based segmentation was used by the authors of [33] to compare the performance of three different algorithms under a single unified framework in performing multi-lesion detection. Out of these, a compacted watershed algorithm was found to be faster, whereas a linear spectral clustering (LSC) algorithm generated better segmentation results. After this, a single-layer CNN was used to detect lesions. Dai et al. [34] introduced a novel system called DeepDR comprising of three sub-networks (based on ResNet and Mask-RCNN) to grade DR using a large dataset collected between 2014 and 2017. Lesion segmentation was performed before DR grading. The Javeria segmentation (JSeg) model, a novel model with the ResNet-50 model as the backbone, was utilized for segmentation by the authors in [35]. In their next phase, Resnet-101 was used for feature extraction built on the equilibrium optimizer (EO). After this, support vector machine (SVM) and neural network classifiers were used for grading lesions. The YOLOv3 model was used for lesion localization by the authors in [36]. It was fused with the CNN512 model to grade DR and localize DR lesions. Wang et al. [37] introduced a new model called adaptive multi-target weighting network (AMWNet) to segment DR lesions. It consisted of an encoder, decoder and an adaptive multi-target weighting module. After this reverse data recovery network (RRN) was used to replicate the cycle perception as part of the hierarchy. DR lesion detection using a CNN-based one-shot detector was used by the authors in [38]. The YOLOv4 deep neural network (DNN) architecture was used along with the Darknet framework.

Many studies reviewed, detected multiple DR lesions. Pixel-wise segmentation of MA, EX, and OD were achieved by the authors of [39] using a novel membrane system termed “dynamic membrane system” and mask R-CNN. MA and EX segmentation was performed by the authors of [40] by using a revised U-Net architecture which was grounded on the residual network. U-Net with incremental learning was used by the authors in [41] for EX and HM segmentation. A new incremental learning standard was introduced to excerpt the acquaintance gained by the preceding model to advance the present model. Another study that detected EX and HM was performed by the authors in [42]. They performed a semantic segmentation with three classes and used a color space transformation along with U-Net. Ananda et al. [43] used one individual network per one type of lesion related to DR (e.g., one network for MA, one network for HE, etc.). They found that this modification yielded better results for U-Net to classify MA, SE, and HM and detect DR. U-Net segmentation followed by classification using support vector machine was achieved by the authors of [44] for grading DR by detecting some DR lesions. Blood vessels, EX, MA, and HM were segmented for this purpose. A nested U-Net architecture known as U-Net++ was used for red lesion segmentation (MA and HM segmentation) from fundus images by the authors in [45]. This was followed by sub-image classification using ResNet-18 to reduce false positives. Another study that focused on red lesion detection was one by the authors of [46]. For this purpose, they fine-tuned various pre-trained CNN models with an augmented set of image patches and found ResNet50 to be the best. They modified it with reinforced skip connections to get a new model called DR-ResNet50.

The authors in [47] used MA, EX, and HM diagnosis to detect DR by using regions with CNNs (R-CNN). Feature extraction using the VGG19 model trialed by classification using different machine learning classifiers by the authors of [48]. They identified and classified DR lesions into MA, SE, HE, and HM. A single-scale CNN (SS-CNN) followed by a multi-scale CNN (MS-CNN) was used by the authors of [49] to perform pixel-level detection of MA and HM. Four DR lesions, including MA, HE, SE, and HM, were segmented by the authors of [50] using a relation transformer network (RTNet). In this, a relation transformer block was incorporated into attention mechanisms in two levels. Another new model was used to detect these same four lesions by the authors in [51]. It was called faster region-based CNN (RCNN) and was a combination of a regional proposal network (RPN) and fast-RCNN. Another new model known as deeply-supervised multiscale attention U-Net (Mult-Attn-U-Net) was used by the authors of [52] for detecting MA, EX, and HM. It was based on the U-Net architecture. Dual-input attentive RefineNet (DARNet) was used by the authors in [53] to detect four DR lesions including HE, SE, HM, and MA. DARNet included a global image encoder, local image encoder, and attention refinement decoder. The same authors used a cascade attentive RefineNet for detecting the same four DR lesions in another study [54]. CARNet included a global image encoder, a patch image encoder, and an attention refinement decoder. A YOLOv5 model along with a genetic algorithm was used by the authors in [55] to detect MA, SE, HE, and HM. Another study by the authors of [56] detected MA, EX, and HM with the help of a CNN along with a SVM. Prior to this, they used a U-Net to segment OD and retinal blood vessels for feature extraction.

Three studies among the reviewed works, focused on the segmentation of the DR lesion called HM. Skouta et al. [57] made use of a U-Net architecture for detecting the presence of HM. Aziz et al. [58] utilized a novel CNN model known as HE network (HemNet) for HM detection. Quality enhancement, seed points extraction, image calibration, and smart window-based adaptive thresholding were performed prior to training this model. A 3D CNN deep learning framework with feature fusion was used to perform HM detection by the authors in [59]. A variation of the traditional CLAHE method was for pre-processing to achieve better edge detection.

Table 1. Deep Learning based approaches for diabetic retinopathy lesion detection.

Type	Method	Year	Technique	Dataset
Exudates	Adem [17]	2018	Adaptive histogram equalization, CNN	DIARETDB0, DIARETDB1
	Raja and Balaji [16]	2019	CNN and fuzzy c-means clustering, SVM	DRIVE, STARE
	Wang et al. [20]	2019	CNN	Self-collected
	Wang et al. [18]	2020	Deep CNN, multi-feature joint representation	E-Optha, DIARETDB1
	Mateen et al. [24]	2020	Inception-v3, ResNet-50 and VGG-19	DIARETDB1, E-ophtha
	Ghosh and Ghosh [12]	2021	rSVM, CNN	STARE, DIARETDB0, DIARETDB1
	Kurilová et al. [19]	2021	SVM, RCNN, ResNet-50	E-ophtha, DIARETDB1, MESSIDOR
	Huang et al. [22]	2021	Patch-based deep CNN	DIARETDB1, E-ophtha, IDRiD
	Bibi et al. [23]	2021	RCNN	DIARETDB1, E-ophtha
	Cincan et al. [25]	2021	SqueezeNet, ResNet50 and GoogLeNet	-
	Sivapriya et al. [21]	2022	RNN	MESSIDOR
	Manan et al. [26]	2022	RTC-net	DIARETDB1, E-ophtha
Microaneurysms	Qiao et al. [28]	2020	Deep CNN	IDRiD
	Qomariah et al. [30]	2021	MResUNet	DIARETDB1, IDRiD
	Xia et al. [27]	2021	MSRNet, MS-EfficientNet	E-Ophtha, IDRiD, DRIVE, CHASE DB1
	Subhasree et al. [31]	2022	VGG-16, Inception V3, and ResNet50	E-Ophtha, DIARETDB1
	Gupta et al. [29]	2022	VGG, Inception V3	ROC
Multiple Lesions	Xiao et al. [32]	2019	cGAN, HEDNet	IDRiD
	Ananda et al. [43]	2019	U-Net, SegNet	IDRiD, MESSIDOR
	Praveena and Lavanya [33]	2019	Super-pixel based segmentation	DIARETDB1, DRiDB, HRF
	Xue et al. [39]	2019	CNN	IDRiD, E-Optha, MESSIDOR
	Gupta et al. [48]	2019	VGG-19	IDRiD
	Nazir et al. [51]	2020	RCNN	DIARETDB1, MESSIDOR
	Sambyal et al. [40]	2020	Modified U-Net architecture	IDRiD, E-Optha
	He et al. [41]	2020	U-Net, Incremental learning	-
	Dai et al. [34]	2021	ResNet, Mask-RCNN	Self-collected
	Alyoubi et al. [36]	2021	YOLOv3, CNN512	DDR
	Wang et al. [37]	2021	AMWNet	IDRiD
	Abdelmaksoud et al. [44]	2021	U-Net, SVM	DRIVE, STARE, CHASEDB1, E-ophtha
	Li et al. [49]	2021	SS-CNN, MS-CNN	DIARETDB1
	Basu and Mitra [52]	2021	Mult-Attn-U-Net	IDRiD
	Santos et al. [38]	2021	YOLOv4 with darknet	DDR
	Amin et al. [35]	2022	JSeg model	IDRiD
	Kundu et al. [45]	2022	U-Net++, ResNet-18	MESSIDOR, DIARETDB1
	Latchoumi et al. [47]	2022	R-CNN	-
	Huang et al. [50]	2022	RTNet	IDRiD, DDR
	Guo and Peng [53]	2022	DARNet	IDRiD, DDR, E-ophtha
	Guo and Peng [54]	2022	CARNet	IDRiD, DDR, E-ophtha
	Ashraf et al. [46]	2022	DR-ResNet50	DIARETDB1, E-ophtha, IDRiD
	Santos et al. [55]	2022	YOLOv5	DDR
	Selçuk et al. [42]	2022	U-Net	MESSIDOR, DIARETDB1
	Jena et al. [56]	2023	U-Net, CNN	APTOS, MESSIDOR
Hemorrhage	Maqsood et al. [59]	2021	3D CNN with feature fusion	DIARETDB0, DIARETDB1, MESSIDOR, HRF, DRIVE, STARE
	Skouta et al. [57]	2022	U-Net	IDRiD, DIARETDB1
	Aziz et al. [58]	2023	HemNet	DIARETDB0, DIARETDB1

Table 1. Deep Learning based approaches for diabetic retinopathy lesion detection.

Type	Method	Year	Technique	Dataset
Exudates	Adem [17]	2018	Adaptive histogram equalization, CNN	DIARETDB0, DIARETDB1
	Raja and Balaji [16]	2019	CNN and fuzzy c-means clustering, SVM	DRIVE, STARE
	Wang et al. [20]	2019	CNN	Self-collected
	Wang et al. [18]	2020	Deep CNN, multi-feature joint representation	E-Optha, DIARETDB1
	Mateen et al. [24]	2020	Inception-v3, ResNet-50 and VGG-19	DIARETDB1, E-ophtha
	Ghosh and Ghosh [12]	2021	rSVM, CNN	STARE, DIARETDB0, DIARETDB1
	Kurilová et al. [19]	2021	SVM, RCNN, ResNet-50	E-ophtha, DIARETDB1, MESSIDOR
	Huang et al. [22]	2021	Patch-based deep CNN	DIARETDB1, E-ophtha, IDRiD
	Bibi et al. [23]	2021	RCNN	DIARETDB1, E-ophtha
	Cincan et al. [25]	2021	SqueezeNet, ResNet50 and GoogLeNet	-
	Sivapriya et al. [21]	2022	RNN	MESSIDOR
	Manan et al. [26]	2022	RTC-net	DIARETDB1, E-ophtha
Microaneurysms	Qiao et al. [28]	2020	Deep CNN	IDRiD
	Qomariah et al. [30]	2021	MResUNet	DIARETDB1, IDRiD
	Xia et al. [27]	2021	MSRNet, MS-EfficientNet	E-Ophtha, IDRiD, DRIVE, CHASE DB1
	Subhasree et al. [31]	2022	VGG-16, Inception V3, and ResNet50	E-Ophtha, DIARETDB1
	Gupta et al. [29]	2022	VGG, Inception V3	ROC
Multiple Lesions	Xiao et al. [32]	2019	cGAN, HEDNet	IDRiD
	Ananda et al. [43]	2019	U-Net, SegNet	IDRiD, MESSIDOR
	Praveena and Lavanya [33]	2019	Super-pixel based segmentation	DIARETDB1, DRiDB, HRF
	Xue et al. [39]	2019	CNN	IDRiD, E-Optha, MESSIDOR
	Gupta et al. [48]	2019	VGG-19	IDRiD
	Nazir et al. [51]	2020	RCNN	DIARETDB1, MESSIDOR
	Sambyal et al. [40]	2020	Modified U-Net architecture	IDRiD, E-Optha
	He et al. [41]	2020	U-Net, Incremental learning	-
	Dai et al. [34]	2021	ResNet, Mask-RCNN	Self-collected
	Alyoubi et al. [36]	2021	YOLOv3, CNN512	DDR
	Wang et al. [37]	2021	AMWNet	IDRiD
	Abdelmaksoud et al. [44]	2021	U-Net, SVM	DRIVE, STARE, CHASEDB1, E-ophtha
	Li et al. [49]	2021	SS-CNN, MS-CNN	DIARETDB1
	Basu and Mitra [52]	2021	Mult-Attn-U-Net	IDRiD
	Santos et al. [38]	2021	YOLOv4 with darknet	DDR
	Amin et al. [35]	2022	JSeg model	IDRiD
	Kundu et al. [45]	2022	U-Net++, ResNet-18	MESSIDOR, DIARETDB1
	Latchoumi et al. [47]	2022	R-CNN	-
	Huang et al. [50]	2022	RTNet	IDRiD, DDR
	Guo and Peng [53]	2022	DARNet	IDRiD, DDR, E-ophtha
	Guo and Peng [54]	2022	CARNet	IDRiD, DDR, E-ophtha
	Ashraf et al. [46]	2022	DR-ResNet50	DIARETDB1, E-ophtha, IDRiD
	Santos et al. [55]	2022	YOLOv5	DDR
	Selçuk et al. [42]	2022	U-Net	MESSIDOR, DIARETDB1
	Jena et al. [56]	2023	U-Net, CNN	APTOS, MESSIDOR
Hemorrhage	Maqsood et al. [59]	2021	3D CNN with feature fusion	DIARETDB0, DIARETDB1, MESSIDOR, HRF, DRIVE, STARE
	Skouta et al. [57]	2022	U-Net	IDRiD, DIARETDB1
	Aziz et al. [58]	2023	HemNet	DIARETDB0, DIARETDB1

2.2. Diabetic Retinopathy Retinal Blood Vessel/Optic Disc Segmentation

This category includes those studies which performed retinal blood vessel segmentation as well as those which performed OD/region of interest (ROI) segmentation. Most studies fall under the first type, which helps to study the vascular tree and can thus help assess the current medical condition of the patient’s eyes. Other studies focused on optic disc (OD)/fovea segmentation and ROI segmentation. These studies can aid the feature extraction process required for developing automatic DR diagnosis systems by helping to remove unnecessary features before doing them [60].

One of the most preferred DL methods for segmentation is to use CNNs [61,62,63,64,65]. The presence of neovascularization was used to spot proliferative DR by the authors in [66]. Different layers of CNNs were modeled together with VGG-16 net architecture for this. They performed blood vessel segmentation to remove the extra features before doing the classification. Chala et al. [67] used a multi-encoder decoder architecture with two encoder units, as well as a decoder unit for retinal blood vessel segmentation. Aujih et al. [68] studied how the U-Net model performs for the same purpose. Dropout and batch normalization with various settings were utilized for this. Batch normalization was found to make U-Net learn faster but degraded the performance after epoch thirty. They also studied how the presence/absence of retinal blood vessel segmentation affected the DR classification performance using Inception v1. Burewar et al. [69] made use of U-Net for retinal segmentation with region merging. After this, CNN with ReLU activation function and max-pooling was utilized to perform DR grading. Yadav [70] applied a dual-tree discrete Ridgelet transform (DT-DRT) to fundus images for feature extraction from ROIs and used a U-Net-based method for retinal blood vessel segmentation after this. The U-Net++ architecture was used to extract the retinal vasculature by the authors in [71]. They used the extracted features to predict DR in the next stage of their experiment. A context-involved U-Net was used to perform retinal blood vessel segmentation in the study by the authors in [72]. They made use of patch-based loss weight mapping to improve the segmentation of thin blood vessels. Another study by the authors in [73] used an encoder enhanced atrous U-Net to extract the retinal vasculature from fundus images. They introduced two additional layers to the encoder at every stage to enhance the extraction of edge information. Jebaseeli et al. [74] utilized contrast limited adaptive histogram equalization (CLAHE) to perform pre-processing. After this, feature generation and classification were performed. The former step was accomplished using the tandem pulse coupled neural network model and the latter was achieved using DL based support vector machine (DLBSVM). DLBSVM also extracted the blood vessel tree from the fundus images. Similar work was done by the authors of [75]. Another study by the authors of [76] made use of a pool-less residual segmentation network (PLRS-Net) for retinal blood vessel segmentation, which could segment smaller vessels better. They used two pool-less approaches called PLRS-Net and PLS-Net (pool-less segmentation network) which did not require any pre/post-processing. A multilevel deep neural network with a three-plane preprocessing method was used by the authors of [77] for retinal blood vessel segmentation. The first four convolutional layers of VGG-16 were fine-tuned for blood vessel feature extraction. Jin et al. [78] performed retinal blood vessel segmentation using a new model known as deformable U-Net (DUNet). This model combined the benefits of the U-Net architecture with those of a deformable convolutional network. Another new network followed, a network called $NFN+$ was used by the authors of [79] to segment retinal blood vessels. A distinct cascaded design, along with inter-network connections made this model unique. Distinctive segmentation of thick as well as thin retinal blood vessels was performed by the authors of [80] by using a three-stage DL model. These stages consisted of the segmentation of thick vessels, the segmentation of thin vessels, and the fusion of vessels. Another model known as multi-scale CNN with attention mechanisms (MSCNN-AM) was used by the authors in [81] to segment retinal blood vessels using various dimensions. Atrous separable convolutions which had differing dilation rates were used to seize global and multi-scale vessel data more effectively. Li et al. [82] used an enhanced U-Net architecture in order to segment retinal blood vessels. They made use of data from the West China Hospital of Sichuan University to perform their experiments. Three DL models consisting of SegNet, U-Net, and CNN were used by the authors of [83] for the same task. The SegNet model was found to be better than the other two models in segmenting both thin and thick blood vessels. A back propagation neural network was utilized by the researchers in [84] to extract the retinal vasculature from fundus images. They could obtain a reduction in operation time in addition to improved accuracy. A pre-processing AlexNet block followed by an encoder-decoder U-Net model was utilized by the authors in [85] to extract the retinal vasculature. Their study aimed to support the diagnosis of several ophthalmological diseases including DR. Another similar study that performed extraction of the retinal vasculature was performed by the authors in [86] and used a multi-scale residual CNN known as MSR-Net, along with a generative adversarial network (GAN). The generator of the GAN was a segmentation network, whereas the discriminator was a classification network. Another GAN called Pix2Pix GAN was used by the authors in [87] for retinal blood vessel segmentation to help studies that detect different ophthalmological diseases. A novel model known as D-MNet with multi-scale attention and a residual mechanism, was used by the authors in [88] for retinal blood vessel segmentation. It was used along with an enhanced version of pulse-coupled neural network to achieve this task.

Some studies performed OD segmentation. This was achieved by using an encoder-decoder with the VGG-16 backbone by the authors of [89]. Convolutional long short term memory was merged into the encoder to improve accuracy. Kumar et al. [90] used a mathematical morphology operation before extracting the retinal blood vessels from fundus images. OD segmentation was done with the help of watershed transform. Finally, classification was done using radial basis function neural network (RBFNN). Yeh et al. [91] used a preprocessing technique for OD segmentation called local differential filter (LDF). Good prediction results were obtained when LDF was used for U-Net model training data to differentiate between OD and background regions. Retinal blood vessel segmentation, as well as extraction of the optic disc as well as fovea centers, were performed by the authors in [92] using an end-to-end encoder-decoder network known as DRNet. Shankar et al. [93] introduced a deep CNN model named Synergic DL model for retinopathy grading. They also performed histogram-based OD segmentation prior to this.

Another set of studies in this category focused on ROI segmentation. An algorithm named “adaptive-learning-rate-based” enhanced GMM was used for ROI segmentation in fundus images by Mansour [94]. After this, they used connected component analysis (CCA) to process the input to AlexNet architecture. This was followed by feature selection and grading using a polynomial kernel-based SVM classifier. Active DL was used in a seven-layer CNN architecture (ADL-CNN) by Qureshi et al. [95] to automatically grade the images into the five stages of DR after retinal blood vessel segmentation as well as ROI localization. Table 2 summarizes the studies that were presented in this section.

Table 2. Deep-learning based approaches for diabetic retinopathy segmentation.

Study	Year	Method	Dataset
Aujih et al. [68]	2018	U-Net, InceptionV1	MESSIDOR
Burewar et al. [69]	2018	U-Net segmentation	DRIVE
Yan et al. [80]	2018	CNN	STARE, DRIVE, CHASE-DB1
Jebaseeli et al. [74]	2019	TPCNN, DLBSVM	STARE, DRIVE, HRF, REVIEW, DRIONS
Li et al. [82]	2019	U-Net	-
Jebaseeli et al. [75]	2019	TPCNN, DLBSVM	STARE, DRIVE, HRF, REVIEW, DRIONS
Samuel and Veeramalai [77]	2019	DNN	STARE, DRIVE, HRF
Jin et al. [78]	2019	DUNet	STARE, DRIVE, HRF, CHASE-DB1
Yeh et al. [91]	2020	U-Net architecture	Self-collected
Kumar et al. [90]	2020	RBFNN	DIARET DB0,DIARET DB1
Shankar et al. [93]	2020	Deep CNN model-SDL	MESSIDOR
Wu et al. [79]	2020	$NFN+$	STARE, DRIVE, CHASE-DB1
Fu et al. [81]	2020	MSCNN-AM	STARE, DRIVE, CHASE-DB1
Hasan et al. [92]	2020	DRNet	IDRiD, HRF, DRIVE
Liu [84]	2021	Back propagation neural network	MESSIDOR, DRIVE
Qureshi et al. [95]	2021	ADL-CNN	EyePACS
Saranya et al. [66]	2021	VGG-16 net architecture	DRIVE, STARE
Chala et al. [67]	2021	Multi-encoder decoder architecture	STARE, DRIVE
Popescu et al. [87]	2021	Pix2Pix GAN	CHASE DB1, DRIVE, STARE
Sathananthavathi and Indumathi [73]	2021	Encoder enhanced atrous U-Net	CHASE DB1, DRIVE, STARE, HRF
Deng and Ye [88]	2022	D-MNet, pulse-coupled neural network	CHASE DB1, DRIVE, STARE, HRF
Yadav [70]	2022	U-Net architecture	CHASE DB1, DRIVE, STARE
Gargari et al. [71]	2022	U-Net++ architecture	DRIVE, MESSIDOR
Zhang et al. [72]	2022	Context-involved U-Net	DRIVE, CHASE-DB1, STARE, HRF
Arsalan et al. [76]	2022	PLRS-Net, PLS-Net	DRIVE, CHASE-DB1, STARE
Maiti et al. [89]	2022	Encoder-decoder with VGG-16	DRIVE, DIARETDB0, CHASE-DB1, STARE
Elaouaber et al. [83]	2022	SegNet, U-Net, CNN	DRIVE, CHASE-DB1, HRF
Prajna and Nath [85]	2022	AlexNet, U-Net	DRIVE, ARIA, MESSIDOR
Kar et al. [86]	2022	MSR-Net, GAN	DRIVE, ARIA, CHASE-DB1, STARE, HRF

3. Datasets Used for Diabetic Retinopathy Segmentation/Lesion Detection

The datasets being employed have a significant role in the accomplishment of the DL studies reviewed here. Some factors like the quality of these datasets, as well as the precision of their annotations, impact the results of such studies. Therefore, it will be useful to have a list of a few generally utilized fundus image datasets for DR segmentation/lesion detection. We have presented such a list in Table 3.

Table 3. Datasets used for Diabetic Retinopathy Segmentation/Lesion Detection.

Dataset	No. of Images	Image Size
DIARETDB0	130	1500 × 1152
DIARETDB1	89	1500 × 1152
STARE	400	700 ×605
DRIVE	40	768 × 584
IDRiD	516	4288 × 2848
E-Ophtha	463	Different sizes
MESSIDOR	1200	Different sizes
HRF	45	3504 × 2336
REVIEW	16	1360 × 1024
DRIONS	110	600× 400
CHASE DB1	28	999 × 960
DDR	13,673	Different sizes
ARIA	143	768 × 576

Some of the generally utilized, publicly accessible datasets in such studies include DRIVE, STARE, IDRiD, E-ophtha, MESSIDOR, DIARET DB1, and DIARET DB0. Images gathered during a DR assessment program in the Netherlands were employed to create the dataset known as Digital Retinal Images for Vessel Extraction (DRIVE). It consists of twenty images each for training and testing sets [96]. The STARE dataset was collected as part of a project known as STARE (Structured Analysis of the Retina). It provides forty hand-labeled images for blood vessel segmentation [97]. The Indian DR Image Dataset (IDRiD) provides groundtruth for DR and Diabetic Macular Edema (DME). It contains annotations for DR grading, DR lesions, and optic disc [98]. The E-Optha dataset comprises of two sub-datasets called E-Optha-MA and E-Optha-EX [99]. Just as their names indicate these datasets have been annotated for EX and MA correspondingly. Another dataset known as DIARETDB1 has been annotated by four medical experts for DR lesions like MA and EX [100]. The MESSIDOR dataset was part of the MESSIDOR project and has been publicly available since 2008 [101]. The DDR dataset was collected from 9598 patients and 757 images with DR have been annotated for DR lesions [102].

Out of these datasets, DRIVE and STARE datasets are the highly applied datasets for retinal blood vessel segmentation. However, since manual annotation of the retinal blood vessels is time consuming and tedious, these datasets have very few images. Datasets like IDRiD, E-Optha MA, DIARET DB0, and DIARET DB1 have been annotated for MA detection. E-Optha EX, IDRiD, DIARET DB0, and DIARET DB1 have been annotated for EX detection. Whereas the annotation for OD is available for MESSIDOR and DIARET DB1 datasets. It was noted that only very few datasets are annotated for retinal blood vessel segmentation and for lesion detection. This may be because marking retinal blood vessels manually will require a lot of time, as well as effort, and annotating lesions demands intra-annotator agreement and high annotation precision. Due to this reason, the majority of the studies utilized more than one dataset for training as well as validation.

4. Discussion

Several DL architectures were used to effectively perform DR segmentation and lesion detection tasks in the studies reviewed in the previous section. Some examples of these architectures/models are ResNet, Inception, EfficientNet, U-Net, and Encoder-Decoder. Apart from these, most studies used different pre-processing techniques for gaining better performance and for image enhancement. This can include different steps like intensity conversion, image variation attenuation, contrast enhancement, and denoising [103]. Since there will be a huge disparity in the retinal color of different patients, attenuation of fundus images may have to be done. Additionally, to make the features in an image distinctly visible, intensity conversion can be used. Another pre-processing step is the denoising of fundus images. This is necessary, since throughout the image acquisition procedure plenty of noise may have been introduced in these images. Lastly, since there will be highest contrast at the image centre for fundus images, which steadily reduces when moving away from the centre, contrast enhancement may have to be performed. Additional pre-processing steps consist of image resizing, grayscale conversion, gaussian filtering, applying CLAHE [74], and doing several image augmentations using techniques like rotation, flip, and zoom. Some morphological operations like erosion, dilation, top-hat transformation, and bottom-hat transformation were accomplished on the green channel images of the fundus images as pre-processing in [90]. Hasan et al. [92] resized and standardized the images as part of pre-processing. They also performed image augmentation and generated 2D heatmaps as pre-processing.

The studies under review can be compared using several metrics like accuracy, sensitivity, and specificity. These are some of the most commonly used metrics in computer vision. This part presents the results attained per dataset by utilizing the cited methods for segmentation or lesion detection. This has been done by reporting the results with the help of tables and figures to identify the best-performing deep-learning architectures for these tasks. Table 4, Table 5 and Table 6 present an evaluation of the results achieved by a few studies which were assessed in this study. A visual comparison of some studies that performed retinal blood vessel segmentation, MA detection, and HE detection have been provided in Figure 4 and Figure 5. Finally, Figure 6 provides a comparison of the results attained by researchers for deep-learning-based HE and MA detection on the IDRiD dataset, which is one of the biggest and most highly used datasets preferred by researchers for DR segmentation/lesion detection.

Table 4. Performance comparison of deep-learning based diabetic retinopathy segmentation.The bold and underlined fonts correspondingly show the first and second place.

Dataset	Method	Accuracy	Sensitivity	Specificity
STARE	Jebaseeli et al. [74]	0.997	0.806	0.997
	Chala et al. [67]	0.965	0.81	0.984
	Yadav [70]	0.986	0.812	-
	Saranya et al. [66]	0.95	0.938	0.99
	Arsalan et al. [76]	0.972	0.864	0.98
	Jebaseeli et al. [75]	0.991	0.707	0.996
	Samuel and Veeramalai [77]	0.965	0.898	0.97
	Jin et al. [78]	0.964	-	-
	Wu et al. [79]	0.967	0.986	0.796
	Yan et al. [80]	0.964	0.774	0.986
	Fu et al. [81]	0.966	0.841	0.981
	Kar et al. [86]	0.949	0.844	0.964
	Popescu et al. [87]	0.915	0.818	0.942
	Zhang et al. [72]	0.967	0.8	0.986
	Sathananthavathi and Indumathi [73]	0.945	0.802	0.956
	Deng and Ye [88]	0.973	0.827	0.985
DRIVE	Jebaseeli et al. [74]	0.99	0.803	0.998
	Chala et al. [67]	0.972	0.821	0.988
	Yadav [70]	0.977	0.821	-
	Hasan et al. [92]	0.999	0.962	-
	Burewar et al. [69]	0.933	-	-
	Arsalan et al. [76]	0.968	0.827	0.982
	Jebaseeli et al. [75]	0.99	0.701	0.995
	Samuel and Veeramalai [77]	0.961	0.828	0.974
	Jin et al. [78]	0.957	-	-
	Wu et al. [79]	0.958	0.799	0.981
	Yan et al. [80]	0.954	0.763	0.982
	Fu et al. [81]	0.956	0.834	0.973
	Elaouaber et al. [83]	0.977	0.967	0.996
	Liu [84]	0.958	0.814	0.906
	Gargari et al. [71]	0.989	0.941	0.988
	Prajna and Nath [85]	0.906	0.918	0.905
	Kar et al. [86]	0.974	0.894	0.988
	Popescu et al. [87]	0.921	0.834	0.961
	Zhang et al. [72]	0.957	0.785	0.982
	Sathananthavathi and Indumathi [73]	0.958	0.792	0.971
	Deng and Ye [88]	0.968	0.836	0.981

Table 5. Performance comparison of deep-learning based diabetic retinopathy segmentation. The bold and underlined fonts correspondingly show the first and second place.

Dataset	Method	Accuracy	Sensitivity	Specificity
REVIEW	Jebaseeli et al. [74]	0.999	0.809	0.988
	Jebaseeli et al. [75]	0.995	0.693	0.989
HRF	Jebaseeli et al. [74]	0.99	0.808	0.997
	Jebaseeli et al. [75]	0.99	0.826	0.994
	Samuel and Veeramalai [77]	0.853	0.866	0.852
	Jin et al. [78]	0.965	-	-
	Elaouaber et al. [83]	0.979	0.979	0.995
	Kar et al. [86]	0.977	0.889	0.985
	Zhang et al. [72]	0.96	0.857	0.969
	Sathananthavathi and Indumathi [73]	0.924	0.659	0.98
	Deng and Ye [88]	0.967	0.754	0.985
DRIONS
	Jebaseeli et al. [74]	0.999	0.805	0.998
	Jebaseeli et al. [75]	0.993	0.795	0.997
CHASE DB1	Yadav [70]	0.96	0.802	-
	Arsalan et al. [76]	0.973	0.83	0.984
	Jin et al. [78]	0.961	-	-
	Wu et al. [79]	0.969	0.8	0.988
	Yan et al. [80]	0.961	0.764	0.981
	Fu et al. [81]	0.964	0.813	0.982
	Elaouaber et al. [83]	0.955	0.837	0.982
	Kar et al. [86]	0.987	0.934	0.984
	Popescu et al. [87]	0.934	0.917	0.969
	Zhang et al. [72]	0.967	0.813	0.984
	Sathananthavathi and Indumathi [73]	0.934	0.646	0.965
	Deng and Ye [88]	0.971	0.854	0.979
IDRiD	Wang et al. [37]	0.983	0.485	0.999
	Hasan et al. [92]	0.997	0.899	-
MESSIDOR	Aujih et al. [68]	0.84	-	-
	Liu [84]	0.962	0.852	0.911
	Prajna and Nath [85]	0.834	0.797	0.844
ARIA	Kar et al. [86]	0.963	0.718	0.984
	Prajna and Nath [85]	0.925	0.566	0.961

Table 6. Performance comparison of deep-learning based diabetic retinopathy lesion-detection. The bold and underlined fonts correspondingly show the first and second place.

Type	Dataset	Method	Accuracy	Sensitivity	Specificity
Exudates	IDRiD	Sambyal et al. [40]	0.997	0.806	0.997
		Abdelmaksoud et al. [44]	1	1	1
	MESSIDOR	Selçuk et al. [42]	0.958	0.987	0.915
		Sivapriya et al. [21]	0.97	0.95	0.99
	E-Optha	Bibi et al. [23]	-	0.96	0.99
		Manan et al. [26]	0.99	0.92	-
		Mateen et al. [24]	0.984	-	-
	DIARET DB1	Abdelmaksoud et al. [44]	0.987	0.97	0.962
		Manan et al. [26]	0.98	0.95	-
		Cincan et al. [25]	0.979	0.96	0.96
		Adem [17]	-	0.992	0.979
		Bibi et al. [23]	-	0.98	0.94
		Manan et al. [26]	0.98	0.95	0.99
		Selçuk et al. [42]	0.949	0.979	0.905
		Mateen et al. [24]	0.989	-	-
Hard Exudates	DDR	Guo and Peng [53]	0.974	0.966	0.978
		Guo and Peng [54]	0.974	0.966	0.978
	E-Optha	Xue et al. [39]	0.984	0.846	0.988
		Wang et al. [18]	-	0.899	-
		Guo and Peng [53]	0.975	0.965	0.985
		Guo and Peng [54]	0.975	0.965	0.985
		Huang et al. [22]	0.976	0.983	0.912
	MESSIDOR	Sivapriya et al. [21]	0.973	0.95	0.99
Microaneurysm	E-Optha	Xue et al. [39]	0.992	0.672	0.998
		Guo and Peng [53]	0.949	0.935	0.958
		Guo and Peng [54]	0.949	0.935	0.958
		Subhasree et al. [31]	0.96	0.9	0.85
		Xia et al. [27]	-	0.740	-
	DDR	Guo and Peng [53]	0.954	0.936	0.965
		Guo and Peng [54]	0.954	0.936	0.965
	DIARET DB1	Abdelmaksoud et al. [44]	0.98	0.96	0.98
		Qomariah et al. [30]	0.997	0.859	0.998
		Li et al. [49]	-	0.88	-
		Subhasree et al. [31]	0.91	0.94	0.88
Hemorrhage	DIARET DB0	Maqsood et al. [59]	0.955	-	-
		Aziz et al. [58]	0.971	0.91	0.98
	DIARET DB1	Maqsood et al. [59]	0.964	-	-
		Selçuk et al. [42]	0.943	0.972	0.896
		Aziz et al. [58]	0.972	0.942	0.975
	MESSIDOR	Maqsood et al. [59]	0.994	-	-
		Selçuk et al. [42]	0.96	0.987	0.917

Figure 4. Retinal blood vessel segmentation results on the DRIVE dataset using [75,76] methods, respectively. Left: Original Image. Middle: Ground-truth, Right: Segmentation Result.

Figure 5. Comparison of deep-learning-based diabetic retinopathy microaneurysm detection and hard exudates detection results performed by [35,43] respectively on the IDRiD dataset.

Figure 6. Performance comparison of deep-learning based diabetic retinopathy with hard exudate and microaneurysm detection studies which employed the IDRiD dataset [22,28,31,33,38,40,41,50,53,54].

4.1. Diabetic Retinopathy Segmentation

DR segmentation-based studies include those studies that segment retinal blood vessels, OD, or ROI from fundus images. Table 4 and Table 5 provides a comparison of the reviewed DR retinal blood vessel segmentation studies grouped according to the datasets used by them. The segmented results obtained will be helpful for extended research in the area of DR. According to these tables, it is obvious that most of the studies obtained good accuracies but the specificity and sensitivity values are unavailable for some studies. It is clear that on the STARE dataset, the authors of [74] attained the highest accuracy value of 99.7% which is better by 0.6% than the second highest accuracy value of 99.1%, obtained by the authors in [75] and 1.1% better than the accuracy values obtained by other methods. They could also achieve the highest specificity value compared to other methods. Whereas, when sensitivity is taken into account, the highest value was obtained by the authors in [66].

When we look at the values obtained for the DRIVE dataset in the table, the same study that achieved the highest accuracy value for the STARE dataset also obtained the best value for specificity as well as the second-best value for accuracy. Whereas, the highest values for accuracy and sensitivity, 99.9% and 96.7% were obtained by the authors in [83,92], respectively. Figure 4 provides a further comparison of retinal blood vessel segmentation results using the DRIVE dataset by the researchers in [75,76]. We can observe that the former study clearly segments the retinal blood vessels without any trace of OD and fovea, whereas in the second study parts of the OD and fovea are present in the obtained result.

One of the major challenges which researchers find while performing retinal blood vessel segmentation, is the segmention of tiny blood vessels. Hence, more studies are required that can effectively detect these similar to the study of [76]. It would also be beneficial to classify the detected blood vessels into arteries/veins as well as to determine the width of these blood vessels, which will provide valuable information that can be used to determine the present condition of the patient’s eyes. When it comes to OD segmentation, the major challenge is that other DR lesions may be wrongly segmented as OD due to their similar shape, size, and color.

4.2. Lesion Detection

DR lesion detection-based studies include studies that detected DR lesions like MA, EX, and HM. This section attempts to compare these studies using three metrics which include accuracy, sensitivity, and specificity. The values for these are stated in many studies. Table 6 provides a comparison of DR lesion-detection-based studies. The studies have been grouped according to the type of lesions they detected as well as according to the datasets used by them. In the case of EX, the maximum number of studies used the DIARET DB1, and the highest value for accuracy was obtained by the authors in [24] which is 0.2% higher than the second-best accuracy value obtained by the authors in [44]. The highest sensitivity value was obtained by the authors in [17], whereas the authors in [26] obtained the highest specificity value. From the table, it can also be seen that the E-Optha dataset has been used by many studies for HE and MA detection. In the first case, the highest accuracy and specificity values of 98.4% and 98.8%, respectively, were achieved by the authors in [39]. They also achieved the best accuracy and specificity values of 99.2% and 99.8%, respectively, for MA detection also. Whereas, the highest values of sensitivity for HE and MA detection on the E-Optha dataset were achieved by the authors in [22,53,54]. In the case of HM, the maximum accuracy of 99.4% on the MESSIDOR dataset was obtained by the authors in [59]. Whereas, the maximum accuracy value of 97.2% on the DIARET DB1 dataset was obtained by the authors in [58].

Figure 5 provides a comparison of MA detection results on the IDRiD dataset by the researchers in [35,43] and a comparison of HE detection results on the IDRiD dataset by the researchers in [35,43]. As seen in the figure, the authors in [43] could effectively detect HE using U-Net and MA using a modified U-Net. Whereas, the authors in [35] could obtain similar results using the JSeg model.

Figure 6 provide a further comparison of the results obtained by researchers for deep-learning-based HE and MA detection on the IDRiD dataset. It can be clearly seen that the authors in Sambyal et al. have obtained the best value of 99.9% for all three metrics for both HE and MA detection on the IDRiD dataset. This is followed by the study by the authors in Xue et al., which achieved very close values for accuracy and specificity. In the case of HE detection, all the studies could achieve accuracy values greater than 98% except for the study by the authors in Gupta et al. [48].

Just like DR segmentation, there are several challenges to performing DR lesion detection. If we take the case of MA detection, the main challenge is their very small size. But MA detection is very important to determine the onset of DR. EX detection is also challenging since they may not have well-defined boundaries. Additionally, as mentioned before, the OD and fovea may be mistakenly detected as lesions due to their similar shape, size, and color. Finally, the size of DR lesions, like EX, vary according to the severity of DR which makes it even more challenging.

5. Future Directions

Diabetic retinopathy is a topic in which a large amount of research is happening lately. In this section, we would like to present some possible directions for future work related to DR segmentation and DR lesion detection.

• The number of studies which performed optic disc/fovea segmentation are relatively few in the reviewed literature. Hence, more research studies are needed in this area.
• Similarly, in the case of DR lesion detection, more studies need to be performed for HM/SE detection.
• Due to the serious complications that can result from proliferative DR, it is desirable to detect DR at an early stage. Hence, it will be beneficial to have more studies which focus on non-proliferative DR lesions.
• Security is of prime importance for any proposed method for DR segmentation/lesion detection. Hence, studies like the recent one by the authors in [104] are required. They have used adversarial training and feature fusion for DR detection.
• DR patients are at the risk of developing other conditions like glaucoma. Hence, more studies related to such conditions similar to the latest study by the authors in [105] are needed. They performed glaucoma detection by using a novel deep CNN.

6. Conclusions

The latest literature in the field of “diabetic retinopathy segmentation and lesion detection from fundus images using deep learning” was surveyed. The studies in this field can be grouped as blood vessel segmentation based studies, lesion detection based studies, and OD segmentation-based studies. Lesion detection-based studies can further be classified into MA detection, EX (soft and hard) detection, and HM detection.

We found that almost all recent DL networks/architectures have been utilized effectively for DR segmentation and lesion detection in the studies that were reviewed. It was also observed that there is a spike in the number of these kinds of studies recently. We also created a table of the generally utilized retinal fundus image datasets for DR retinal blood vessel segmentation and lesion detection. Finally, we performed a comparison based on the performance of studies from each type of DR study reviewed here. As future work, we may do a systematic literature review in this field.