Next Article in Journal
A Phasor Measurement Unit Algorithm Using IIR Filters for FPGA Implementation
Previous Article in Journal
Constant Envelope Modulation Techniques for Limited Power Millimeter Wave Links
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Accelerating Retinal Fundus Image Classification Using Artificial Neural Networks (ANNs) and Reconfigurable Hardware (FPGA)

1
School of Computing, Electronics and Maths, Coventry University, Coventry CV15FB, UK
2
Faculty Research Centre, Institute for Future Transport and Cities, Coventry University, Coventry CV15FB, UK
3
School of Engineering and the Built Environment, Edinburgh Napier University, Edinburgh EH10 5DT, UK
4
Mark Roberts Motion Control, Unit 3, South East Studios, Blindley Heath RH7 6JP, UK
5
Intel, 50-100 Holmers Farm Way, High Wycombe HP12 4PU, UK
6
Faculty of Engineering and Informatics, University of Bradford, Bradford BD7 1DP, UK
7
Information and Communication Engineering Department, Basrah University College of Science and Technology, Basrah 24001, Iraq
*
Author to whom correspondence should be addressed.
Electronics 2019, 8(12), 1522; https://doi.org/10.3390/electronics8121522
Submission received: 30 September 2019 / Revised: 8 December 2019 / Accepted: 9 December 2019 / Published: 11 December 2019
(This article belongs to the Section Computer Science & Engineering)

Abstract

:
Diabetic retinopathy (DR) and glaucoma are common eye diseases that affect a blood vessel in the retina and are two of the leading causes of vision loss around the world. Glaucoma is a common eye condition where the optic nerve that connects the eye to the brain becomes damaged, whereas DR is a complication of diabetes caused by high blood sugar levels damaging the back of the eye. In order to produce an accurate and early diagnosis, an extremely high number of retinal images needs to be processed. Given the required computational complexity of image processing algorithms and the need for high-performance architectures, this paper proposes and demonstrates the use of fully parallel field programmable gate arrays (FPGAs) to overcome the burden of real-time computing in conventional software architectures. The experimental results achieved through software implementation were validated on an FPGA device. The results showed a remarkable improvement in terms of computational speed and power consumption. This paper presents various preprocessing methods to analyse fundus images, which can serve as a diagnostic tool for detection of glaucoma and diabetic retinopathy. In the proposed adaptive thresholding-based preprocessing method, features were selected by calculating the area of the segmented optic disk, which was further classified using a feedforward neural network (NN). The analysis was carried out using feature extraction through existing methodologies such as adaptive thresholding, histogram and wavelet transform. Results obtained through these methods were quantified to obtain optimum performance in terms of classification accuracy. The proposed hardware implementation outperforms existing methods and offers a significant improvement in terms of computational speed and power consumption.

1. Introduction

Healthcare engineering plays an important role in improving the quality of human life. Portable healthcare devices are widely used in diagnosing serious healthcare conditions. Glaucoma and diabetic retinopathy (DR) are two of the serious and widespread eye-related diseases. As reported by the World Health Organization (WHO), diabetes will become the major cause of death by 2030 [1]. It was reported that by 2030, people with diabetes will be 82 million in developing countries and 48 million in developed countries. Similarly, if untreated, DR, which exists in diabetic patients, could lead to blindness in elderly people all over the world, whereas glaucoma, a chronic disease that affects the optic nerve, is the second major cause of blindness leading to loss in the visual field that eventually leads to permanent blindness [2,3]. There are serious risk factors associated with glaucoma, such as raised intraocular pressure that destroys the blood vessels and optic nerve. The ophthalmologists who conduct the tests to detect eye diseases have to be very skilled given the time required to make diagnosis. In the case of DR, there are no symptoms at the early stage and the changes in vision are not noticeable. As the disease progresses, the symptoms of DR, including floating spots in the vision, blurred vision and sudden loss of vision, could appear. In DR, the blood vessels in the retina are eventually affected, and vision is lost. As a result, an over-accumulation of glucose could permanently damage the tiny blood vessels. The main signs of DR are microaneurysms (MAs), hard exudates, haemorrhage, cotton–wool spots, macular oedema, venous loops and venous beading [4], as shown in Figure 1. It was estimated by the International Diabetes Federation (IDF) that the number of diabetic patients will rise up to 592 million by 2035 [5]. As predicted by the WHO, glaucoma is the second foremost cause of blindness worldwide. It was shown in numerous studies that eye pressure is a major risk factor for optic nerve damage, which in turn results in glaucoma. At the front of the eye is a space called the anterior chamber [6], where a clear fluid flows continuously in and out of the chamber and nourishes nearby tissues. The fluid leaves the chamber at the open angle where the cornea and iris meet. When the fluid reaches the angle, it flows through a spongy meshwork which acts as a drain and leaves the eye-opening [3]. A normal fundus image in comparison to glaucoma is shown in Figure 2. In a recent study conducted by the National Eye Institute (NEI) [7], the statistical prediction for the number of people who will be affected by glaucoma is staggering in comparison to previous years.
Hence, identifying these eye diseases at an early stage is essential; however, a large number of patients remains uninformed of the illness until it has progressed to a much more advanced stage [3]. Considering a significant rise in patients suffering from glaucoma and DR, it is the need of the hour that an efficient automated eye disease detection system is developed in both software and hardware that are able to detect glaucoma and DR at early stages so that early treatment could prevent permanent blindness.
The diagnosis of glaucoma and DR mainly involves advanced imaging techniques such as optical coherence tomography (OCT) and scanning laser polarimetry (SLP). However, these methods are expensive and require specialised skills. The fundus images have been widely used to diagnose glaucoma [8,9] and DR [7] where the damage to the optic nerve is detected using features such as the cup to disc ratio and the ratio of distance between the optic disc centre and optic nerve [8]. However, such methods require precision and may not yield good classification accuracy if due attention was not paid to aspects such as suitable feature selection. There are other techniques such as image segmentation that have also been used for glaucoma and DR detection [8]. However, there are a number of drawbacks associated with the image segmentation technique as well, such as thresholding and localisation, which may lead to inferior results in diagnosis.
A number of studies have been reported in the literature, such as that of Ravishankaer et al. [10] and others [11,12,13,14,15,16], which have shown that retinal blood vessels could be accurately detected in the images using image processing algorithms. Sinthanayothin et al. [11] applied moat operators to the green channel of the enhanced image. In retrospect, red findings were extracted with thresholding.
The principle behind these algorithms is to first detect blood vessels and then approximate the location of the optic disk. However, segmentation algorithms have proven to be complex and time-consuming. There are a number of techniques reported in the literature, such as in [8], where the authors proposed a method where a non-uniform fundus image intensity was estimated and shade correction was performed by enhancing the green channel of the image and subtracting the result from the original fundus image. Another image enhancement method was adopted by [17], where colours were normalised by applying the histogram. Principal component analysis (PCA) was used to reduce the feature size, and a support vector machine (SVM) was used for classification. In order to detect the bright DR areas from fundus images, Zhang and Chutape [17] applied adaptive contrast enhancement using the local mean and standard deviation of intensities.
Furthermore, deep convolutional neural networks (DCNN) have successfully been applied to analyse images [18]; however, despite the attractive qualities of DCNN and the relative efficiency of their local architectures, there are a number of drawbacks associated with this, such as high computational cost due to high-resolution images (such as fundus images) and the need for large training datasets.
In the context of the wider literature and state-of-the-art, this paper offers two major contributions—improved classification accuracy and hardware acceleration through field programmable gate arrays (FPGAs). Firstly, an in-depth analysis of various feature selection methods is provided, and the most suitable method is proposed for the detection of glaucoma and DR images by means of a rather simple and lightweight adaptive thresholding method. In this method, the main emphasis is given on localising the optic disk and cup. This method is shown to be more effective, less compute-intensive and simpler. A second major contribution of this paper is the acceleration and implementation of the neural classifier on the hardware platform (FPGA). Hardware implementation is particularly important to demonstrate the viability of classifying fundus images in real time. Hardware implementation is also important to demonstrate a route from a proof of concept design to a successful prototype. Given the amount of time required to simulate large datasets in software, it is essential that neural architectures be implemented on a fully parallel hardware platform. This study demonstrates the viability of such mapping from the software domain to fully parallel reconfigurable hardware (FPGA).
This paper is organised as follows:
The Materials and Methods section describes the proposed methodology in both software and hardware. The Results and Discussion section provides an insight into the test design and experimental analysis. The Conclusion section is devoted to concluding remarks with future work followed by References.

2. Materials and Methods

In this section, various preprocessing techniques are analysed, and an adaptive thresholding technique is proposed to localise the optic disk and hard exudates for both glaucoma and DR images. Other techniques such as histogram and wavelet transform were also investigated, and results are shown. In order to carry out the experiments and formulate the data sets for the classification of glaucoma, the optic disk was first segmented from the fundus image. The high-resolution fundus (HRF) image database was used for the preprocessing methods. The database can be downloaded from the public website: ‘https://www5.cs.fau.de/research/data/fundus-images/’. This dataset consists of three sets of retinal fundus images: healthy retinas, glaucoma images and DR retinas. Each set contains 15 images each, whereas healthy images include healthy retinas, DR images include images of patients with haemorrhages, bright lesions and spots after laser treatment and, finally, the glaucoma images include patients suffering from symptoms of focal and diffuse nerve fibre layer loss. Each image belongs to a different patient. The images were divided into training and test, and the neural classifier was tested with the unseen test images. In total, 10 images each for glaucoma, healthy and DR were used for training and 5 images each for glaucoma, healthy and DR were used for testing. All images were acquired with a CANON EOS– 20D digital camera with a 60-degree field of view (FOV) [19].
The images were independently classified by three trained ophthalmologists in retinal imaging and were labelled. Once the fundus images were preprocessed, glaucoma and DR classification were done using the feed-forward backend neural classifier. For all the digital fundus images, a corresponding ground truth file was included. The total dataset was divided into a training and test dataset.
In the following section, different preprocessing methods were investigated for feature selection and results are reported.

2.1. Preprocessing Based on Histogram

Histograms are generally used to represent the statistical relationship between the number of grey levels and the occurrence frequency of a digital image [13]. In order to assess the suitability, the histogram approach is used to localise the optic disk from the retinal fundus image. Each fundus image is represented by a collection of histograms, extracted with respect to the RGB (red, green and blue) and HSI (hue, saturation and intensity) colour models. As shown in Figure 3a (ii and iv), it is difficult to differentiate between optic disk and optic cup in blue and red channel images; however, the green channel as shown in Figure 3a (iii) provides much better contrast, and it is relatively easy to locate the boundary of both optic disk and the optic cup. In order to reduce the total number of histograms, the only green channel was used for preprocessing. The green channel was selected as shown to offer the best discriminatory power between the main retinal anatomy (blood vessels, fovea and optic disk) and the retinal background [20]. The RGB channels of retinal fundus image are shown in Figure 3a, and the corresponding histogram of the selected green channel representation is shown in Figure 3b, where pixel intensity is shown in the x-axis and pixel count is shown in the y-axis. Once the histogram is analysed, the optic disk and optic cup are segmented by providing a threshold value that has a high intensity in the histogram; as can be seen from the fundus image, the optic disk and cup are the brighter regions of the image.
Mathematically, in a histogram, the probabilities assigned to each grey level can be given by the relation shown in Equation (1):
p r r k =   n k N 0 r k 1 ,   k = 0 , 1 , 2 L 1
where rk corresponds to the normalised intensity value, L to the number of grey levels in the image, nk to the number of pixels with grey level rk and N to the total number of pixels, whereas the plot of pr (rk) with respect to rk is the histogram of the image. Once the histogram of the image is obtained, then the maximum index values are selected to localise the optic disk, since the optic disk is the brightest part in the fundus image. The optic disk is located with a red ‘+’ sign as shown in Figure 4c. Once the optic disk is localised, then the region of interest is segmented from that to formulate the data set for backend classification. The optic disk segmentation process is represented in Figure 4c,d. As shown in Figure 4, an original RGB fundus image (a) was converted into the corresponding grey image of the green channel (Figure 4b). As shown in Figure 4c, the optic disk is located by means of finding the maximum intensity pixel. Since the optic disk is of high intensity, by finding the maximum intensity pixel, the optic disk can be exactly located. A red’+’ sign is used as an indicator to point that location. Once the optic disk is localised, the optic disk is segmented from the green channel grey image as shown in the zoomed view of Figure 4d.

2.2. Preprocessing Based on Adaptive Thresholding

Adaptive thresholding is one of the most robust and efficient methods for segmenting an image [21]. Since the region of interest (ROI) is the optic disk, the method of adaptive thresholding can be used effectively to segment the optic disk from the fundus image. In adaptive thresholding, the analysis is done for the distributed pixel intensity in the fundus image [22], and a threshold value is selected. The optic disk can be segmented by the mathematical representation as given in Equation (2). Here, the input image is denoted by g(x,y), I is the threshold value and output image is denoted by f(x,y). This expression can be written as:
f x , y =     1   i f     g x , y I 0   o t h e r w i s e
In this particular example, an analysis was done on the pixel intensity distribution where the value of I is chosen as 150. Using this method, the following results are obtained for the segmentation of optic disk as shown in Figure 5, where Figure 5b shows a zoomed view.
The abovementioned methodology was implemented to localise the hard exudates from the fundus image as a part of feature selection for the classification of the DR. The pixel intensity distribution is analysed and the value of I is chosen as 75 < I < 120. The following results were obtained for the localisation of hard exudates as shown in Figure 6, whereas Figure 6b shows a zoomed view of the image shown in Figure 6a.

2.3. Preprocessing Based on Discrete Wavelet Transform (DWT)

In this method, DWT was applied on a green channel of the fundus image. The transformed image is decomposed using Daubechies wavelet ‘db2’ having two vanishing moments. Daubechies wavelet ‘db2’ was selected for the decomposition of the green component image because it is one of the conditions to segment the optic disk by excluding other bright lesions in the retinal image. The use of higher-order wavelet filters increases the number of computations, which in turn increases the computational complexity without much improvement. Hence, in order to reduce the complexity and computations with good efficiency, Daubechies wavelet ‘db2’ having two vanishing moments were selected for the decomposition of the fundus image [23]. The general equation of two dimensional forward and inverse discrete wavelet transforms for an image f(x,y) is given in Equations (3)–(6).
A j j 0 , m , n = 1 M N x = 0 M 1 y = 0 N 1 f x , y Ψ j 0 , m , n x , y
D i j 0 , m , n = 1 M N x = 0 M 1 y = 0 N 1 f x , y Ψ j 0 , m , n x , y
i = H , V , D
f x , y = 1 M N m n A j j 0 , m , n Ψ j 0 , m , n x , y +   1 M N i = H , V , D j = 0 m n D i j 0 , m , n   Ψ j 0 , m , n x , y
The two-dimensional wavelet transform shown in the above equations leads to a decomposition of approximation coefficients at level j-1 in four components: the approximations at level j, i.e., Aj (LL) and the details in three orientations (horizontal (H), vertical (V) and diagonal (D)), i.e., H (LH), V (HL) and D (HH). Once the wavelet transform is implemented in the image to sort the brighter segment, an optic disk can be segmented, and the data sets can be formulated. The optic disk segmentation using wavelet transform is represented in Figure 7.

2.4. Backend Recognition Using a Neural Classifier

Once the data were preprocessed, the feed-forward neural classifier was used to classify glaucoma from healthy images and then classifying glaucoma from the DR images. In total, 15 images were taken for testing, 5 images each for glaucoma, DR and healthy images [19,24]. There have been several studies reported in the literature for classifying retinal images with various preprocessing methods in combination with machine learning techniques; a comprehensive review and details are provided in [25,26,27,28]. As stated in [28], Niemeijer et al. used the probability of finding pixels by Gaussian filters. For classification, shape and intensity properties were used.
In this paper, different neural network (NN) architectures were implemented for the classification of healthy, glaucoma and DR images. The best results were achieved by implementing the architecture as shown in Figure 8.
The flowchart which illustrates the classification process is shown in Figure 9.

2.5. Hardware Implementation

In order to improve the computational time and exploit the parallelism of neural networks, the hardware implementation was performed on an FPGA device (Nexys4 DDR) to classify the healthy, glaucoma and DR images. The neural architecture shown in Figure 8 was implemented by using Very High-speed Hardware Description Language (VHDL) structural coding on the Artix-7 chip [29]. Since the preprocessing step was performed in MATLAB for the classification of glaucoma, DR and healthy images, only the area of the segmented optic disk was fed into the FPGA device through a universal asynchronous receiver and transmitter (UART) as a 16-bit input. In order to realise the implementation on hardware, different blocks were designed and developed. A UART was configured, which is an integrated circuit to control a computer’s interface to its attached serial device. In this study, the input from the MATLAB has to be fed into the FPGA; hence, a UART receiver was modelled. The data were sent bit-by-bit, and only one bit at a time was fed into the intermediate register. The stream of data was organised such that there was a start bit followed by the collective serial data bits. Once the data were fully transmitted and received, they were shifted into the intermediate register of the receiver and provided the 16-bit serial data, which was given as an input to the transmitter. The UART fundamentally provides an interface with the RS-232C data terminal equipment (DTE).
The block diagram shown in Figure 10 represents the proposed hardware design for implementation.
Figure 10 represents the proposed neural network architecture implemented on a Xilinx Artix FPGA device. The neural network classifier contains one hidden layer and an output layer. The individual components, such as a multiplier, adder, UART and sigmoid activation function, were modelled separately, and the entire neural network structure was implemented using structural modelling in VHDL. Once the design was functionally verified, it was synthesised, and the bitstream was uploaded on the FPGA device. A schematic diagram of the hardware neural architecture is shown in Figure 11, and the synthesised design on-chip is shown in Figure 12. Power dissipation on the hardware platform mainly depends on the way hardware components are designed and connected, whereas the computational speed depends on the routing signals between various components and calculating the worst-case propagation delay of the circuit.
In order to implement the software-based neural classifier onto hardware, it is important that this design is carefully planned to both determine the most suitable hardware structure as well as the best mappings from the neural-network structures onto given hardware structures. As neural networks are inherently parallel, one of the biggest advantages of using an FPGA device is to exploit this parallelism by fully mapping the software architecture [30,31]. The hardware implementation results include behavioural simulations, elaborated RTL design and the proof of implementation on the physical FPGA. Further results are elaborated on in the following section.

3. Results and Discussion

3.1. Software Implementation Results

The feature sets were formulated from the abovementioned sections (Section 2.1, Section 2.2 and Section 2.3), where three different preprocessing techniques were evaluated. Out of the three preprocessing techniques implemented for the classification of glaucoma, DR and healthy images, the datasets formulated using the method with adaptive thresholding gave the best overall accuracy. The training procedure included 10 images each for glaucoma, DR and healthy, where features were selected as the area of the OD for glaucoma and area of the total exudates for DR. The accuracy between glaucoma and healthy images is plotted in Figure 13a and accuracy between glaucoma and DR images as a regression plot is shown in Figure 13b. The training accuracy for both glaucoma and healthy images and glaucoma and DR images in terms of convergence is shown in Figure 14a,b, respectively.
The classification accuracy using the data set formulated by an adaptive thresholding-based preprocessing technique was recorded as 99.901% for glaucoma and healthy images, as shown in Figure 13a. For the classification of glaucoma and DR images, the classification accuracy is recorded as 99.623%. The classification accuracy in terms of regression is plotted in Figure 13b.
In total, five fundus image samples from each class were taken for testing, and the relevant confusion matrices are shown in Figure 15. As shown in Figure 15, the correctly classified samples can be identified in the diagonal directions of the confusion matrix marked in the green squares. As the binary classification was performed, the network classification results were achieved with 100% accuracy. The ANN [32] classifier results showed overall accuracies of ~100% for the training and test data sets.
With all the 10 samples tested, the experimental result matches the ground truth. The evaluation of the performance in terms of computational time is shown in Figure 16.
As shown in Figure 16, the computational time for each sample varied from 1.2–2 s both for DR and glaucoma images. A comparative study with regard to classification accuracy is shown in Table 1. As shown in Table 1, the authors of [22] reported that in total, 20 samples were used to test glaucoma and normal images. In [33], authors used, in total, 81 samples for testing, which included 31 images of healthy retina and 50 images of the diseased retina. In [31], the authors did not report total test images; however, the algorithm was evaluated on all 1200 images. In the proposed method, in total, 15 samples were used for testing healthy, glaucoma and DR images.
As shown in Table 1, the proposed method offers better accuracy in comparison to the previously published techniques. It is due to adopting a rather simple and better feature selection method (adaptive thresholding) combined with a backend neural architecture. In the proposed adaptive thresholding based preprocessing method, the calculation of the area of the segmented optic disk was incorporated rather than calculating the local image features, such as mean and standard deviation.

3.2. Hardware Implementation Results

The hardware design was physically downloaded onto the FPGA device, whereas the output pin from the board was connected to an LED on a breadboard. The LED turns ON/OFF corresponding to the classified output as glaucoma or a healthy image. The hardware setup is shown in Figure 17.
As shown in Figure 17, after the preprocessing step, when a 16-bit input obtained from the MATLAB is fed into the hardware where the neural classifier was implemented, the LED remains OFF to show that the input image is healthy; however, when an image is classified with glaucoma, the LED turns ON.
The computation time is an important factor to consider when large datasets are classified. As shown in Figure 16, the computation time to classify each image as either glaucoma or healthy ranges from 1.2 to 2 seconds in the software domain. Once the neural classifier was implemented on hardware, the total computation time was recorded through behavioural simulations, as shown in Figure 18. As shown in Figure 18, two sample values were fed to the hardware device within a timeframe of 800 ns. Hence, it takes approximately 400 ns for one sample to be classified. This shows that the hardware implementation is 3 × 106 times faster than software implementation. For this specific architecture, the logic utilisation in terms of lookup tables (LUT) is shown in Table 2. In total, 3.23% logic was utilised, which included the UART (0.38%), Sigmoid function (2.835%) and adder (0.02%). The total power consumption (static/dynamic) is shown in Table 3, where on-chip power was recorded as 13.776 W.

3.3. Experimental Setup

An automated retinal fundus image classification was implemented on both software and hardware platforms. The software implementation was performed in MATLAB on an Intel Core i5-6200 CPU. The hardware implementation was realised on a Nexys4 DDR FPGA board (Artix-7). The hardware design was developed using Xilinx Vivado Design Suite [34], whereas the power was measured using Xilinx Synthesis Tools. Table 4 reports the performance of software and hardware implementations in terms of execution time and power consumption. The software implementation in MATLAB took 1.2–2 s to compute each sample, whereas the average power consumption was recorded as 25.656 W. Similarly, FPGA implementation took almost 400 ns to compute one sample, and average power was recorded as 9.434 W. As can be seen, the hardware implementation is almost 3 x 106 times faster and almost 2 times more power-efficient compared to the software. The proposed work offers a significant improvement in terms of classification accuracy and execution time when compared to the existing work (Table 5). The proposed work offers significantly improved performance in comparison to the work reported in [35,36,37].

4. Conclusions

Automatic retinal image classification is a challenging task, and devising a preprocessing methodology along with backend classification is crucial to accurately diagnose diseases such as glaucoma and DR from healthy images. The authors proposed an adaptive thresholding-based preprocessing technique which has proven to be one of the most robust methodologies. As the adaptive thresholding process involves calculating the area of the segmented optic disk, it does not depend on factors such as intensity, contrast, etc. This work demonstrated the viability of achieving optimum performance in terms of classification accuracy, where 100% accuracy was achieved with both glaucoma and healthy images and glaucoma and DR images. In order to speed up the execution time, hardware implementation was realised on an FPGA device. In this study, it was demonstrated that high accuracy and performance could be achieved without compromising on precision. This study also offers a unique perspective that FPGAs could be used for real-time diagnosis in healthcare, where large datasets are used and output is required in real time. In future work, we would extend this work to develop a standalone low-power hand-held custom design device that could perform retinal image classification in real time.

Author Contributions

Conceptualisation, A.G.; Formal analysis, V.S.; Investigation, A.G. and V.S.; Project administration, A.G.; Resources, C.H.S., J.A. and R.A.-A.; Supervision, A.G.; Writing—original draft, A.G.; Writing—review and editing, C.H.S, V.S, A.G.

Funding

This research received no external funding.

Acknowledgments

Coventry University’s Faculty Research Centre on Institute for Future Transport and Cities, UK.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. World Health Organization. Global Prevalence of Diabetes. Available online: https://www.who.int/diabetes/facts/en/diabcare0504.pdf (accessed on 20 October 2019).
  2. Khalid, S.; Akram, M.S.; Hassan, T.; Nasim, A.; Jameel, A. Fully Automated Robust System to Detect Retinal Edema, Central Serous Chorioretinopathy, and Age Related Macular Degeneration from Optical Coherence Tomography Images. BioMed Res. Int. 2017, 2017, 7148245. [Google Scholar] [CrossRef]
  3. Mary, M.C.V.S.; Rajsingh, E.B.; Naik, G.R. Retinal Fundus Image Analysis for Diagnosis of Glaucoma: A Comprehensive Survey. IEEE Access 2016, 4, 4327–4354. [Google Scholar] [CrossRef]
  4. Karami, N.; Rabbani, H. A dictionary learning based method for detection of diabetic retinopathy in color fundus images. In Proceedings of the 10th Iranian Conference on Machine Vision and Image Processing (MVIP), Isfahan, Iran, 22–23 November 2017; pp. 119–122. [Google Scholar]
  5. Zhou, L.; Zhao, Y.; Yang, J.; Yu, Q.; Xu, X. Deep multiple instance learning for automatic detection of diabetic retinopathy in retinal images. IET Image Process. 2018, 12, 563–571. [Google Scholar] [CrossRef]
  6. Chang, S.H.L.; Lee, Y.; Wu, S.; See, C.C.; Yang, M.; Lai, C.; Wu, W. Anterior Chamber Angle and Anterior Segment Structure of Eyes in Children with Early Stages of Retinopathy of Prematurity. Am. J. Ophthalmol. 2017, 179, 46–54. [Google Scholar] [CrossRef] [PubMed]
  7. National Eye Institute. Available online: https://nei.nih.gov/ (accessed on 20 October 2019).
  8. Kohler, T.; Budai, A.; Kraus, M.F.; Odstrcilik, J.; Michelson, G.; Hornegger, J. Automatic no-reference quality assessment for retinal fundus images using vessel segmentation. In Proceedings of the 26th IEEE International Symposium on Computer-Based Medical Systems, Porto, Portugal, 20–22 June 2013; Volume 1, pp. 95–100. [Google Scholar]
  9. Bharkad, S.D. Automatic Segmentation of Optic Disk in Retinal Images Using DWT. In Proceedings of the 2016 IEEE 6th International Conference on Advanced Computing (IACC), Bhimavaram, India, 27–28 February 2016; Volume 1, pp. 100–106. [Google Scholar] [CrossRef]
  10. Ravishankar, S.; Jain, A.; Mittal, A. Automated feature extraction for early detection of diabetic retinopathy in fundus images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA, 20–25 June 2009; pp. 210–217. [Google Scholar]
  11. Sinthanayothin, C.; Boyce, J.F.; Williamson, T.H.; Cook, H.L.; Mensah, E.; Lal, S.; Usher, D. Automated detection of diabetic retinopathy on digital fundus images. Diabet. Med. 2002, 19, 105–112. [Google Scholar] [CrossRef] [Green Version]
  12. Faust, O.; Acharya, R.U.; Ng, E.Y.K.; Ng, K.; Suri, J.S. Algorithms for the automated detection of diabetic retinopathy using digital fundus images: A review. J. Med. Syst. 2012, 36, 145–157. [Google Scholar] [CrossRef]
  13. Nayak, J.; Bhat, P.S.; Acharya, R.; Lim, C.M.; Kagathi, M. Automated identification of diabetic retinopathy stages using digital fundus image. J. Med. Syst. 2008, 32, 107–115. [Google Scholar] [CrossRef]
  14. Ramaswamy, M.; Anitha, D.; Kuppamal, S.P.; Sudha, R.; Mon, S.F.A. A study and comparison of automated techniques for exudate detection using digital fundus images of human eye: A review for early identification of diabetic retinopathy. Int. J. Comput. Technol. Appl. 2011, 2, 1503–1516. [Google Scholar]
  15. Acharya, U.R.; Lim, C.M.; Ng, E.Y.K.; Chee, C.; Tamura, T. Computer-based detection of diabetes retinopathy stages using digital fundus images. Proc. Inst. Mech. Eng. 2009, 223, 545–553. [Google Scholar] [CrossRef]
  16. Hansen, A.B.; Hartvig, N.V.; Jensen, M.S.J.; Borch-Johnsen, K.; Lund-Andersen, H.; Larsen, M. Diabetic retinopathy screening using digital non-mydriatic fundus photography and automated image analysis. Acta Ophthalmol. Scand. 2004, 82, 666–672. [Google Scholar] [CrossRef]
  17. Zhang, X.; Chutape, O. A SVM approach for detection of hemorrhages in background diabetic retinopathy. In Proceedings of the International Joint Conference on Neural Networks, Montreal, QC, Canada, 31 July–4 August 2005; pp. 2435–2440. [Google Scholar]
  18. Kahaki, S.M.; Nordin, M.J.; Ahmad, N.S.; Arzoky, M.; Ismail, W. Deep convolutional neural network designed for age assessment based on orthopantomography data. Neural Comput. Appl. 2019. [Google Scholar] [CrossRef]
  19. Odstrcilik, J.; Kolar, R.; Budai, A.; Hornegger, J.; Jan, J.; Gazarek, J.; Kubena, T.; Cernosek, P.; Svoboda, O.; Angelopoulou, E. Retinal vessel segmentation by improved matched filtering: Evaluation on a new high-resolution fundus image database. IET Image Process. 2013, 7, 373–378. [Google Scholar] [CrossRef]
  20. Mudassar, A.A.; Butt, S. Extraction of Blood Vessels in Retinal Images Using Four Different Techniques. J. Med. Eng. 2013, 2013, 408120. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  21. Bradley, D.; Roth, G. Adaptive Thresholding Using the Integral Image. 2005. Available online: http://www.scs.carleton.ca/~roth/iit-publications-iti/docs/gerh-50002.pdf (accessed on 25 October 2019).
  22. Agarwal, A.; Gulia, S.; Chaudhary, S.; Dutta, M.K.; Burget, R.; Riha, K. Automatic glaucoma detection using adaptive threshold based technique in fundus image. In Proceedings of the 2015 38th International Conference on Telecommunications and Signal Processing (TSP), Prague, Czech Republic, 9–11 July 2015; pp. 416–420. [Google Scholar]
  23. Yun, W.L.; Acharya, U.R.; Venkatesh, Y.; Chee, C.; Min, L.C.; Ng, E. Identification of different stages of diabetic retinopathy using retinal optical images. Inf. Sci. 2008, 178, 106–121. [Google Scholar] [CrossRef]
  24. Magoulas, G.D.; Prentza, A. Machine Learning and Its Applications Lecture Notes in Computer Science; Springer: Berlin/Heidelberg, Germany, 2011; pp. 300–307. [Google Scholar]
  25. El-Sappagh, S.; Elmogy, M.; Ali, F.; Abuhmed, T.; Islam, S.M.; Kwak, K.S. A Comprehensive Medical Decision–Support Framework Based on a Heterogeneous Ensemble Classifier for Diabetes Prediction. Electronics 2019, 8, 635. [Google Scholar] [CrossRef] [Green Version]
  26. Ashfaq, M.; Minallah, N.; Ullah, Z.; Ahmad, A.M.; Saeed, A.; Hafeez, A. Performance Analysis of Low-Level and High-Level Intuitive Features for Melanoma Detection. Electronics 2019, 8, 672. [Google Scholar] [CrossRef] [Green Version]
  27. Vununu, C.; Lee, S.H.; Kwon, K.R. A Deep Feature Extraction Method for HEp-2 Cell Image Classification. Electronics 2019, 8, 20. [Google Scholar] [CrossRef] [Green Version]
  28. Niemeijer, M.; van Ginneken, B.; Staal, J.; Suttorp-Schulten, M.S.A.; Abramoff, N.D. Automatic detection of red lesion in digital color fundus photographs. IEEE Trans. Med. Imaging 2005, 24, 584–592. [Google Scholar] [CrossRef] [Green Version]
  29. Xilinx. Available online: https://www.xilinx.com/products/silicon-devices/fpga/artix-7.html (accessed on 23 October 2019).
  30. Ide, A.N.; Saito, J.H. FPGA Implimentations of Neocognitrons; Springer: Berlin/Heidelberg, Germany, 2006; pp. 197–224. [Google Scholar]
  31. Aquino, A.; Gegúndez-Arias, M.E.; Marín, D. Detecting the optic disc boundary in digital fundus images using morphological, edge, detection, and feature extraction techniques. IEEE Trans. Med. Imaging 2010, 29, 1860–1869. [Google Scholar] [CrossRef] [Green Version]
  32. Kahaki, S.M.M.; Nordin, M.J.; Ismail, W.; Zahra, S.J.; Hassan, R. Blood cancer cell classification based on geometric mean transform and dissimilarity metrics. Pertanika J. Sci. Technol. 2017, 25, 223–234. [Google Scholar]
  33. Hoover, A.; Goldbaum, M. Locating the optic nerve in a retinal image using the fuzzy convergence of the blood vessels. IEEE Trans. Med. Imaging 2003, 22, 951–958. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Xilinx. Available online: https://www.xilinx.com/products/design-tools/vivado.html (accessed on 20 October 2019).
  35. Nieto, A.; Brea, V.M.; Vilariño, D.L. FPGA-accelerated retinal vessel-tree extraction. In Proceedings of the 2009 IEEE International Conference on Field Programmable Logic and Applications, Prague, Czech Republic, 31 August–2 September 2009; pp. 485–488. [Google Scholar]
  36. Koukounis, D.; Tttofis, C.; Theocharides, T. Hardware acceleration of retinal blood vasculature segmentation. In Proceedings of the 23rd ACM International Conference on Great Lakes Symposium on VLSI, Paris, France, 2–4 May 2013; pp. 113–118. [Google Scholar]
  37. Cavinato, L.; Fidone, I.; Bacis, M.; Del Sozzo, E.; Durelli, G.C.; Santambrogio, M.D. Software implementation and hardware acceleration of retinal vessel segmentation for diabetic retinopathy screening tests. In Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju Island, Korea, 11–15 July 2017; pp. 1–4. [Google Scholar]
Figure 1. Retinal fundus images show a difference between a normal retina (a) and diabetic retinopathy (b) where blood vessels are swollen and leak [3].
Figure 1. Retinal fundus images show a difference between a normal retina (a) and diabetic retinopathy (b) where blood vessels are swollen and leak [3].
Electronics 08 01522 g001
Figure 2. Raw fundus images: (a) normal retinal image; (b) glaucoma with poor vascular supply and a cupped optic nerve [3].
Figure 2. Raw fundus images: (a) normal retinal image; (b) glaucoma with poor vascular supply and a cupped optic nerve [3].
Electronics 08 01522 g002
Figure 3. (a) Different channels of fundus images are shown where (i) shows a raw fundus image and (ii) shows a blue channel and (iii) and (iv) show green and red channels, respectively; (b) shows the histogram of red, green and blue (RGB) channels with respect to the pixel intensity and pixel count.
Figure 3. (a) Different channels of fundus images are shown where (i) shows a raw fundus image and (ii) shows a blue channel and (iii) and (iv) show green and red channels, respectively; (b) shows the histogram of red, green and blue (RGB) channels with respect to the pixel intensity and pixel count.
Electronics 08 01522 g003
Figure 4. Feature extraction where the raw image is shown in (a), extraction of green component (b), localisation of the Optic Disk (OD) (c) and finally segmenting the OD to calculate the area (d).
Figure 4. Feature extraction where the raw image is shown in (a), extraction of green component (b), localisation of the Optic Disk (OD) (c) and finally segmenting the OD to calculate the area (d).
Electronics 08 01522 g004
Figure 5. This figure shows the original fundus image (a) and the corresponding OD segmentation using adaptive thresholding (b).
Figure 5. This figure shows the original fundus image (a) and the corresponding OD segmentation using adaptive thresholding (b).
Electronics 08 01522 g005
Figure 6. This figure shows a diabetic retinopathy (DR) image (a) and corresponding hard exudates segmentation using adaptive thresholding, a zoomed view is shown in (b).
Figure 6. This figure shows a diabetic retinopathy (DR) image (a) and corresponding hard exudates segmentation using adaptive thresholding, a zoomed view is shown in (b).
Electronics 08 01522 g006
Figure 7. This figure shows the OD segmentation using the wavelet transform. (a) fundus image and (b) segmented OD.
Figure 7. This figure shows the OD segmentation using the wavelet transform. (a) fundus image and (b) segmented OD.
Electronics 08 01522 g007
Figure 8. Neural network (NN) architecture for glaucoma, DR and healthy image classification.
Figure 8. Neural network (NN) architecture for glaucoma, DR and healthy image classification.
Electronics 08 01522 g008
Figure 9. Classification flowchart.
Figure 9. Classification flowchart.
Electronics 08 01522 g009
Figure 10. Neural network classifier hardware implementation design where input from MATLAB is processed through the hardware architecture implemented on a field programmable gate array (FPGA) device.
Figure 10. Neural network classifier hardware implementation design where input from MATLAB is processed through the hardware architecture implemented on a field programmable gate array (FPGA) device.
Electronics 08 01522 g010
Figure 11. Schematic diagram of a neural architecture implemented on a Nexys4DDR FPGA device.
Figure 11. Schematic diagram of a neural architecture implemented on a Nexys4DDR FPGA device.
Electronics 08 01522 g011
Figure 12. Neural architecture synthesised design on-chip. This figure shows the synthesised design on an Artix 7 FPGA board where embedded blocks such as a multiplier, UART, sigmoid function and adder are shown. The interconnecting wires/signals are shown in green.
Figure 12. Neural architecture synthesised design on-chip. This figure shows the synthesised design on an Artix 7 FPGA board where embedded blocks such as a multiplier, UART, sigmoid function and adder are shown. The interconnecting wires/signals are shown in green.
Electronics 08 01522 g012
Figure 13. Training accuracy using adaptive thresholding for classification of glaucoma and healthy images (a) and glaucoma and DR images (b).
Figure 13. Training accuracy using adaptive thresholding for classification of glaucoma and healthy images (a) and glaucoma and DR images (b).
Electronics 08 01522 g013
Figure 14. (a) shows the convergence where after five epochs, the training converged to an accuracy of 100% for glaucoma and healthy images. (b) Training accuracy for glaucoma and DR images where after 15 epochs, the training converged to 100%.
Figure 14. (a) shows the convergence where after five epochs, the training converged to an accuracy of 100% for glaucoma and healthy images. (b) Training accuracy for glaucoma and DR images where after 15 epochs, the training converged to 100%.
Electronics 08 01522 g014
Figure 15. Confusion matrix for test images on glaucoma and healthy images (a) and glaucoma and DR (b).
Figure 15. Confusion matrix for test images on glaucoma and healthy images (a) and glaucoma and DR (b).
Electronics 08 01522 g015
Figure 16. Computation times for each sample.
Figure 16. Computation times for each sample.
Electronics 08 01522 g016
Figure 17. Classification of a healthy image.
Figure 17. Classification of a healthy image.
Electronics 08 01522 g017
Figure 18. Behavioural simulation of glaucoma and healthy image classification.
Figure 18. Behavioural simulation of glaucoma and healthy image classification.
Electronics 08 01522 g018
Table 1. This table shows a classification accuracy of the proposed method in comparison to the existing methods.
Table 1. This table shows a classification accuracy of the proposed method in comparison to the existing methods.
ReferenceMethodologyFeatures UsedTest Cases AnalysedAccuracy
[31]Edge detectionCircular Hough transformNot reported
(segmentation algorithm applied on all images)
86% accuracy
[33]Fuzzy convergence of blood vesselsBlood Vessel convergence illumination equalisation81 samples for both healthy and diseased retina89% accuracy
[22]Histogram analysis, adaptive thresholdingMean, standard
deviation and cup-disk ratio
20 samples in total for glaucoma and healthy images90% accuracy
Proposed MethodAdaptive ThresholdingArea of the segmented OD15 samples for glaucoma, healthy and DR100% accuracy
Table 2. Logic utilisation breakdown ((Nexys4 DDR xc7a100tcsg324-1)).
Table 2. Logic utilisation breakdown ((Nexys4 DDR xc7a100tcsg324-1)).
Logic OperatorSlice (LUT) 63400
Total utilisation2048 (3.23%)
Adder10 (0.02%)
Embedded Multiplier0
Sigmoid1797 (2.835)
UART243 (0.38%)
Table 3. Power consumption.
Table 3. Power consumption.
Total On-Chip Power13.776 W
Static Power Consumption0.293 W (2% of total)
Dynamic Power Consumption13.484 W (98% of total)
Table 4. Performance metrics between software and hardware platforms.
Table 4. Performance metrics between software and hardware platforms.
PlatformPC/DeviceExecution TimePower Consumption
SoftwareIntel i5-62001.2 s/sample25.656 W
HardwareNexys4 DDR400 ns/sample11.776 W
Table 5. Hardware performance in terms of classification accuracy and execution time.
Table 5. Hardware performance in terms of classification accuracy and execution time.
HardwareClassification AccuracyExecution Time
[35]91%0.4 - 2 s/sample
[36]90%0.03 s/sample
[37]92%0.0104 s/sample
Proposed100%400 ns/sample

Share and Cite

MDPI and ACS Style

Ghani, A.; See, C.H.; Sudhakaran, V.; Ahmad, J.; Abd-Alhameed, R. Accelerating Retinal Fundus Image Classification Using Artificial Neural Networks (ANNs) and Reconfigurable Hardware (FPGA). Electronics 2019, 8, 1522. https://doi.org/10.3390/electronics8121522

AMA Style

Ghani A, See CH, Sudhakaran V, Ahmad J, Abd-Alhameed R. Accelerating Retinal Fundus Image Classification Using Artificial Neural Networks (ANNs) and Reconfigurable Hardware (FPGA). Electronics. 2019; 8(12):1522. https://doi.org/10.3390/electronics8121522

Chicago/Turabian Style

Ghani, Arfan, Chan H. See, Vaisakh Sudhakaran, Jahanzeb Ahmad, and Raed Abd-Alhameed. 2019. "Accelerating Retinal Fundus Image Classification Using Artificial Neural Networks (ANNs) and Reconfigurable Hardware (FPGA)" Electronics 8, no. 12: 1522. https://doi.org/10.3390/electronics8121522

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop