# Information Extraction from Retinal Images with Agent-Based Technology

^{1}

^{2}

^{a}Planta, Paseo de San Vicente, 58-182, 37007 Salamanca, Spain

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Fundus Image Analysis

## 3. Virtual Organizations

## 4. Automatic Image Analyzer to Assess Retinal Vessel Caliber (ALTAIR): A Virtual Organization Based Fundus Image Analyzer Platform

#### 4.1. Virtual Organizations

#### 4.2. Information Extraction Methodology

#### 4.2.1. Image Selection (Eye Side Detection)

Algorithm 1: Eye side detection |

// w and h are the image I width and heightInput: $I,w,h$Output: $S\subseteq I$// ${I}_{ij}\in \mathbb{N}\times \mathbb{N}\times \mathbb{N}$${I}^{1}\leftarrow \{{I}_{ij}/i\ge w\times 0.2,i\le w\times 0.5;j\ge h\times 0.35,j\le h\times 0.65\}$; ${I}^{2}\leftarrow \{{I}_{ij}/i\ge w\times 0.5,i\le w\times 0.8;j\ge h\times 0.35,j\le h\times 0.65\}$; // ${G}^{1}\in \mathbb{N},{G}^{2}\in \mathbb{N}$${G}^{1}\leftarrow rgb2gray\left({I}^{1}\right)$; ${G}^{2}\leftarrow rgb2gray\left({I}^{2}\right)$; $\overline{{G}^{1}}\leftarrow \frac{1}{i\times j}\sum _{\begin{array}{c}i,j\end{array}}{G}_{ij}^{1}$; $\overline{{G}^{2}}\leftarrow \frac{1}{i\times j}\sum _{\begin{array}{c}i,j\end{array}}{G}_{ij}^{2}$; if $\overline{{G}^{1}}>\overline{{G}^{2}}$ then$\phantom{(}$| $S\leftarrow {I}^{1}$; else| $S\leftarrow {I}^{2}$; end |

#### 4.2.2. Bounds Detection

Algorithm 2: Retina edge detection |

// w, h and d are the image I width, height and diameterInput: $I,w,h$Output: d$G\leftarrow rgb2gray\left(I\right)$; // c is the background color, $nc$ is the number of c color point// ${d}_{1}$ is the initial diameter point, ${d}_{2}$ is the end diameter point$c\leftarrow {G}_{00}$; $nc\leftarrow 0$; $d\leftarrow 0$; ${d}_{1}\leftarrow 0$; ${d}_{2}\leftarrow 0$; for $i\leftarrow 1$ to w doend |

Algorithm 3: Optic disc location |

// $crx$ is the x coord of the retina center, $cry$ is the y coord of the retina center and d is the diameter of the retina$\phantom{(}$// $cp$ is the x,y coords of the papillaInput: $I,crx,cry,d$Output: $cp$// ${I}_{ij}\in \mathbb{N}\times \mathbb{N}\times \mathbb{N},{G}_{ij}\in \mathbb{N}$$G\leftarrow rgb2gray\left(I\right)$; $S\leftarrow squareInscribed(G,d/2,crx,cry)$; // Papilla size is always about 6 times lower than the the retina$P\leftarrow \cup {G}_{ij}$; ${P}^{\prime}\leftarrow shortToLower\left(P\right)$; $threshold\leftarrow {P}_{\pi \times {(d/6)}^{2}}^{\prime}$; $B\leftarrow binaryFilter(S,threshold)$; // Open filter to erode and dilate the image B$O\leftarrow openFilter\left(B\right)$; // Fill gaps in the image O$\phantom{(}$$C\leftarrow closeFilter\left(O\right)$; $blob\leftarrow getBiggestBlob\left(C\right)$; $rect\leftarrow getBoundingRect\left(blob\right)$; $cp\leftarrow getCenter\left(rect\right)$; |

#### 4.2.3. Segmentation

Algorithm 4: Vessels detection |

// $cpx$ is the x coord of the papilla center, $cpy$ is the y coord of the papilla center and $rp$ is the radio of the papilla// ${D}^{\prime}$ is image with the detected vessels in white and the other points in blackInput: $I,cpx,cpy,rp$Output: ${D}^{\prime}$// ${I}_{ij}\in \mathbb{N}\times \mathbb{N}\times \mathbb{N},I{r}_{ij}\in \mathbb{N}$$Ir\leftarrow rgb2gray(I,infraredFilter)$; // P contains all the papilla points$P\leftarrow \{I{r}_{ij}/{(i-cpx)}^{2}+{(j-cpy)}^{2}\le r{p}^{2}\}$; $S\leftarrow \{I{r}_{ij}/i\ge cpx-rp\times 1.5,i\le cpx+rp\times 1.5;j\ge cpy-rp\times 1.5,j\le cpy+rp\times 1.5\}$; ${S}^{\prime}\leftarrow {S}_{ij}/S\cup P=\varnothing $; // Change papilla color with the closest background mean color to avoid noise when applying gaussian filters$P\leftarrow \overline{{S}^{\prime}}$; // Analysis area goes from the papilla limit to 3 times its radio size$A\leftarrow \{I{r}_{ij}/i\ge cpx-rp\times 4,i\le cpx+rp\times 4;j\ge cpy-rp\times 4,j\le cpy+rp\times 4\}$; // $G{a}_{ij}\in \mathbb{N}$$Ga\leftarrow gaussianFilter\left(A\right)$; $I{r}^{\prime}\leftarrow \{I{r}_{ij}/i<cpx-rp\times 4,i>cpx+rp\times 4;j<cpy-rp\times 4,jcpy+rp\times 4\}$; $I{r}^{\prime}\leftarrow \{{A}_{ij}/i\ge cpx-rp\times 4,i\le cpx+rp\times 4;j\ge cpy-rp\times 4,j\le cpy+rp\times 4\}$; // ${D}_{ij}\in \mathbb{N}$$D\leftarrow Ir$; // Grayscale: 0 black; 255 white$\phantom{(}$$threshold\leftarrow I{r}^{\prime}-I{r}^{\prime}/100$; ${D}_{ij}\leftarrow 255/{D}_{ij}\ge threshold$ $\forall ij$; ${D}_{ij}\leftarrow 0/{D}_{ij}<threshold$ $\forall ij$; ${D}_{ij}\leftarrow 0/{P}_{ij}\in P$; // Remove noise: blobs whose area is lower than papilla diameter (all vessels must be bigger)$B\leftarrow getBlobs\left(D\right)$; ${B}^{\prime}\leftarrow \{{B}_{i}/area\left({B}_{i}\right)<(rp\times 2)\}$; ${D}^{\prime}\leftarrow D$; ${D}_{ij}^{\prime}\leftarrow 0/ij\in {B}_{k}^{\prime}$; |

Algorithm 5: Skeleton and segment extraction |

// $\phantom{(}$$cpx$ is the x coord of the papilla center, $cpy$ is the y coord of the papilla center and $rp$ is the radio of the papilla$\phantom{(}$// $\phantom{(}$$Ir$ is the infrared-grayscaled image ${D}^{\prime}$ is a binary image with the detected vessels in white and the other points in black$\phantom{(}$// $\phantom{(}$$Sk$ is a binary image with the vessels skeleton and $S{k}^{\prime}$ is the image with the vessels skeleton represented by the $Ir$ image background points$\phantom{(}$// $\phantom{(}$$TS$ is a subset with the skeleton of all the segments$\phantom{(}$Input: $Ir,{D}^{\prime},cpx,cpy,rp$Output: $Sk,S{k}^{\prime},TS$// $\phantom{(}$$I{r}_{ij}\in \mathbb{N},{D}_{ij}^{\prime}\in \{0,255\}$$\phantom{(}$// $\phantom{(}$A contains all the points of the analyzed area$\phantom{(}$$A\leftarrow \{(i,j)/{(i-cpx)}^{2}+{(j-cpy)}^{2}>r{p}^{2},{(i-cpx)}^{2}+{(j-cpy)}^{2}\le {(rp\times 4)}^{2}\}$; // $\phantom{(}$Get skeleton image ($Sk$) and a new one from it ($S{k}^{\prime}$) with the closest (distance of $rp/2$) background mean instead of white$\phantom{(}$// $\phantom{(}$$S{k}_{ij}\in \{0,255\},S{k}_{ij}^{\prime}\in \mathbb{N}$$\phantom{(}$$Sk\leftarrow getSkeleton\left({D}^{\prime}\right)$; // $\phantom{(}$${P}_{ij}$ contains the set of background points of the retina at a distance of $rp/2$ $\phantom{(}$${P}_{ij}\leftarrow \{(x,y)/{(x-i)}^{2}+{(y-j)}^{2}={(rp/2)}^{2},S{k}_{ij}=255,{D}_{xy}^{\prime}=0,(x,y)\in A\}$; $S{k}^{\prime}\leftarrow Sk$; $S{k}_{ij}^{\prime}\leftarrow \frac{1}{|{P}_{ij}|}\sum {P}_{ij}\forall (i,j)/S{k}_{ij}=255$; // $\phantom{(}$Get all white points type from $Sk$ image (based on the number of neighbors)$\phantom{(}$// $\phantom{(}$$S{P}_{ij}\in \{final,regular,branch,intersection\}$$\phantom{(}$$SP\leftarrow \{(i,j)/S{k}_{ij}=255\}$; $S{P}_{ij}\leftarrow getPointsType\left(S{k}_{ij}\right)$; // $\phantom{(}$See Figure 5$\phantom{(}$// $\phantom{(}$Get all segments $T{S}_{i}\subseteq A$$\phantom{(}$$TS\leftarrow getSegments\left(SP\right)$; |

#### 4.2.4. Identification

#### 4.3. Diagnosis

Algorithm 6: Vessels and segments identification |

// $\phantom{(}$A is the set of segments which have been identified as arteries, V is the set of segments which have been identified as arteries$\phantom{(}$Input: $Ir,S{k}^{\prime},TS,$Output: $A,V$$avThr\leftarrow 15;mxThr\leftarrow 8;count\leftarrow 0;A\leftarrow \left\{\right\};V\leftarrow \left\{\right\}$; while $count<7$ doend |

## 5. Results

## 6. Conclusions

## Author Contributions

## Funding

## Conflicts of Interest

## References

- Sánchez, A.; Villarrubia, G.; Zato, C.; Rodríguez, S.; Chamoso, P. A gateway protocol based on FIPA-ACL for the new agent platform PANGEA. In Trends in Practical Applications of Agents and Multiagent Systems; Springer: Cham, Switzerland, 2013; pp. 41–51. [Google Scholar]
- Macía, I.; Graña, M.; Paloc, C. Knowledge management in image-based analysis of blood vessel structures. Knowl. Inf. Syst.
**2012**, 30, 457–491. [Google Scholar] [CrossRef] - Cheung, C.Y.L.; Ikram, M.K.; Sabanayagam, C.; Wong, T.Y. Retinal microvasculature as a model to study the manifestations of hypertension. Hypertension
**2012**, 60, 1094–1103. [Google Scholar] [CrossRef] [PubMed] - Nguyen, T.T.; Wang, J.J.; Sharrett, A.R.; Islam, F.A.; Klein, R.; Klein, B.E.; Cotch, M.F.; Wong, T.Y. Relationship of Retinal Vascular Caliber with Diabetes and Retinopathy the Multi-Ethnic Study of Atherosclerosis (MESA). Diabetes Care
**2008**, 31, 544–549. [Google Scholar] [CrossRef] [PubMed] - Ortega, M.; Barreira, N.; Novo, J.; Penedo, M.G.; Pose-Reino, A.; Gómez-Ulla, F. Sirius: A web-based system for retinal image analysis. Int. J. Med. Inform.
**2010**, 79, 722–732. [Google Scholar] [CrossRef] [PubMed] - Sanchez, C.I.; Hornero, R.; López, M.I.; Aboy, M.; Poza, J.; Abasolo, D. A novel automatic image processing algorithm for detection of hard exudates based on retinal image analysis. Med. Eng. Phys.
**2008**, 30, 350–357. [Google Scholar] [CrossRef] [PubMed][Green Version] - Ege, B.M.; Hejlesen, O.K.; Larsen, O.V.; Møller, K.; Jennings, B.; Kerr, D.; Cavan, D.A. Screening for diabetic retinopathy using computer based image analysis and statistical classification. Comput. Methods Programs Biomed.
**2000**, 62, 165–175. [Google Scholar] [CrossRef] - Martinez-Perez, M.E.; Hughes, A.D.; Thom, S.A.; Bharath, A.A.; Parker, K.H. Segmentation of blood vessels from red-free and fluorescein retinal images. Med. Image Anal.
**2007**, 11, 47–61. [Google Scholar] [CrossRef] - Podoleanu, A.G.; Rosen, R.B. Combinations of techniques in imaging the retina with high resolution. Prog. Retin. Eye Res.
**2008**, 27, 464–499. [Google Scholar] [CrossRef] - García-Ortiz, L.; Recio-Rodríguez, J.I.; Parra-Sanchez, J.; Elena, L.J.G.; Patino-Alonso, M.C.; Agudo-Conde, C.; Rodríguez-Sánchez, E.; Gómez-Marcos, M.A. A new tool to assess retinal vessel caliber. Reliability and validity of measures and their relationship with cardiovascular risk. J. Hypertens.
**2012**, 30, 770–777. [Google Scholar] [CrossRef] - Lau, Q.P.; Lee, M.L.; Hsu, W.; Wong, T.Y.; Ng, E.Y.K.; Acharya, U.R.; Campillo, A.; Suri, J.S. The singapore eye vessel assessment system. Image Anal. Model. Ophthalmol.
**2014**, 143–160. [Google Scholar] [CrossRef] - Perez-Rovira, A.; MacGillivray, T.; Trucco, E.; Chin, K.S.; Zutis, K.; Lupascu, C.; Tegolo, D.; Giachetti, A.; Wilson, P.; Doney, A.; et al. VAMPIRE: Vessel assessment and measurement platform for images of the REtina. In Proceedings of the 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston, MA, USA, 30 August–3 September 2011; pp. 3391–3394. [Google Scholar]
- GeethaRamani, R.; Balasubramanian, L. Retinal blood vessel segmentation employing image processing and data mining techniques for computerized retinal image analysis. Biocybern. Biomed. Eng.
**2016**, 36, 102–118. [Google Scholar] [CrossRef] - Alberola, J.M.; del Val, E.; Costa, A.; Novais, P.; Julian, V. A genetic algorithm for group formation in elderly communities. AI Commun.
**2018**, 31, 409–425. [Google Scholar] [CrossRef] - Garcia-Ortiz, L.; Perez-Ramos, H.; Chamoso-Santos, P.; Recio-Rodriguez, J.I.; Garcia-Garcia, A.; Maderuelo-Fernandez, J.A.; Gomez-Sanchez, L.; Martínez-Perez, P.; Rodriguez-Martin, C.; De Cabo-Laso, A.; et al. Automatic Image Analyzer to Assess Retinal Vessel Caliber (altair) Tool Validation for the Analysis of Retinal Vessels. J. Hypertens.
**2016**, 34, e160. [Google Scholar] [CrossRef] - Garcia-Ortiz, L.; Gómez-Marcos, M.A.; Recio-Rodríguez, J.I.; Maderuelo-Fernández, J.A.; Chamoso-Santos, P.; Rodríguez-González, S.; de Paz-Santana, J.F.; Merchan-Cifuentes, M.A.; Agudo-Conde, C. Validation of the automatic image analyser to assess retinal vessel calibre (ALTAIR): A prospective study protocol. BMJ Open
**2014**, 4, e006144. [Google Scholar] [CrossRef] [PubMed] - Akil, H.; Huang, A.S.; Francis, B.A.; Sadda, S.R.; Chopra, V. Retinal vessel density from optical coherence tomography angiography to differentiate early glaucoma, pre-perimetric glaucoma and normal eyes. PLoS ONE
**2017**, 12, e0170476. [Google Scholar] [CrossRef] [PubMed] - Tanabe, Y.; Kawasaki, R.; Wang, J.J.; Wong, T.Y.; Mitchell, P.; Daimon, M.; Oizumi, T.; Kato, T.; Kawata, S.; Kayama, T.; et al. Retinal arteriolar narrowing predicts 5 year risk of hypertension in Japanese people: The Funagata Study. Microcirculation
**2010**, 17, 94–102. [Google Scholar] [CrossRef] [PubMed] - Wong, T.Y.; Duncan, B.B.; Golden, S.H.; Klein, R.; Couper, D.J.; Klein, B.E.; Hubbard, L.D.; Sharrett, A.; Schmidt, M.I. Associations between the metabolic syndrome and retinal microvascular signs: The Atherosclerosis Risk in Communities study. Investig. Ophthalmol. Vis. Sci.
**2004**, 45, 2949–2954. [Google Scholar] [CrossRef] - Tikellis, G.; Arnett, D.K.; Skelton, T.N.; Taylor, H.W.; Klein, R.; Couper, D.J.; Richey Sharrett, A.; Wong, T.Y. Retinal arteriolar narrowing and left ventricular hypertrophy in African Americans. The Atherosclerosis Risk in Communities (ARIC) study. Am. J. Hypertens.
**2008**, 21, 352–359. [Google Scholar] [CrossRef] - Yatsuya, H.; Folsom, A.R.; Wong, T.Y.; Klein, R.; Klein, B.E.; Sharrett, A.R. Retinal microvascular abnormalities and risk of lacunar stroke atherosclerosis risk in communities study. Stroke
**2010**, 41, 1349–1355. [Google Scholar] [CrossRef] - Wong, T.Y.; Klein, R.; Sharrett, A.R.; Duncan, B.B.; Couper, D.J.; Tielsch, J.M.; Klein, B.E.; Hubbard, L.D. Retinal arteriolar narrowing and risk of coronary heart disease in men and women: The Atherosclerosis Risk in Communities Study. JAMA
**2002**, 287, 1153–1159. [Google Scholar] [CrossRef] - Daxer, A. The fractal geometry of proliferative diabetic retinopathy: Implications for the diagnosis and the process of retinal vasculogenesis. Curr. Eye Res.
**1993**, 12, 1103–1109. [Google Scholar] [CrossRef] [PubMed] - Mainster, M.A. The fractal properties of retinal vessels: Embryological and clinical implications. Eye
**1990**, 4, 235–241. [Google Scholar] [CrossRef] [PubMed][Green Version] - Tu, S.; Huang, Y.; Liu, G. CSFL: A novel unsupervised convolution neural network approach for visual pattern classification. AI Commun.
**2017**, 30, 311–324. [Google Scholar] - Chapman, N.; Witt, N.; Gao, X.; Bharath, A.A.; Stanton, A.V.; Thom, S.A.; Hughes, A.D. Computer algorithms for the automated measurement of retinal arteriolar diameters. Br. J. Ophthalmol.
**2001**, 85, 74–79. [Google Scholar] [CrossRef] [PubMed][Green Version] - Matsopoulos, G.K.; Mouravliansky, N.A.; Delibasis, K.K.; Nikita, K.S. Automatic retinal image registration scheme using global optimization techniques. IEEE Trans. Inf. Technol. Biomed.
**1999**, 3, 47–60. [Google Scholar] [CrossRef] [PubMed][Green Version] - Zana, F.; Klein, J.C. A multimodal registration algorithm of eye fundus images using vessels detec-tion and Hough transform. IEEE Trans. Med. Imaging
**1999**, 18, 419–428. [Google Scholar] [CrossRef] [PubMed] - Zana, F.; Klein, J.C. Robust segmentation of vessels from retinal angiography. In Proceedings of the 13th International Conference on Digital Signal Processing Proceedings, Santorini, Greece, 2–4 July 1997; Volume 2, pp. 1087–1090. [Google Scholar]
- Espona, L.; Carreira, M.J.; Ortega, M.; Penedo, M.G. A snake for retinal vessel segmentation. In Iberian Conference on Pattern Recognition and Image Analysis; Springer: Berlin/Heidelberg, Germany, 2007; pp. 178–185. [Google Scholar]
- Zhu, P.; Huang, F.; Lin, F.; Li, Q.; Yuan, Y.; Gao, Z.; Chen, F. The relationship of retinal vessel diameters and fractal dimensions with blood pressure and cardiovascular risk factors. PLoS ONE
**2014**, 9, e106551. [Google Scholar] [CrossRef] - Chamoso, P.; Pérez-Ramos, H.; García-García, A. ALTAIR: Supervised Methodology to Obtain Retinal Vessels Caliber. Adv. Distrib. Comput. Artif. Intell. J.
**2014**, 3, 48–57. [Google Scholar] [CrossRef] - Winder, R.J.; Morrow, P.J.; McRitchie, I.N.; Bailie, J.R.; Hart, P.M. Algorithms for digital image processing in diabetic retinopathy. Comput. Med. Imaging Graph.
**2009**, 33, 608–622. [Google Scholar] [CrossRef] - Chamoso, P.; De Paz, J.F.; De La Prieta, F.; Bajo Pérez, J. Agreement technologies applied to transmission towers maintenance. AI Commun.
**2017**, 30, 83–98. [Google Scholar] [CrossRef] - Mena, J.B. Vectorización automática de una imagen binaria mediante K-means y degeneración de la triangulación de Delaunay. Revista de la Asociación Espanola de Teledetección
**2002**, 7, 21–29. [Google Scholar] - Franklin, S.W.; Rajan, S.E. Computerized screening of diabetic retinopathy employing blood vessel segmentation in retinal images. Biocybern. Biomed. Eng.
**2014**, 34, 117–124. [Google Scholar] [CrossRef] - Lam, B.S.; Gao, Y.; Liew, A.W.C. General retinal vessel segmentation using regularization-based multiconcavity modeling. IEEE Trans. Med. Imaging
**2010**, 29, 1369–1381. [Google Scholar] [CrossRef] [PubMed] - Chaudhuri, S.; Chatterjee, S.; Katz, N.; Nelson, M.; Goldbaum, M. Detection of blood vessels in retinal images using two-dimensional matched filters. IEEE Trans. Med. Imaging
**1989**, 8, 263–269. [Google Scholar] [CrossRef] [PubMed]

**Figure 6.**Four kind of points detected when analyzing the vessels’ skeleton. (

**a**) normal; (

**b**) branch; (

**c**) final; (

**d**) cross.

Success (%) /Stage | Eye Side | Retina | Papilla | Segment. | Ident. |
---|---|---|---|---|---|

Proposed System | 100% | 100% | 99% | 95% | 91% |

GeethaRaman et al. (2016) [13] | - | - | - | 95.36% | - |

Franklin et al. (2014) [36] | - | - | - | 95.03% | - |

Lam et al. (2010) [37] | - | - | - | 94.72% | - |

Chaudhuri et al. (1989) [38] | - | - | - | 87.73% | - |

**Table 2.**Main exported parameters for the image Figure 8, right.

vThck | aThck | vArea | aArea | vLength | aLength | AVIndex |
---|---|---|---|---|---|---|

0.119359 | 0.0804724 | 1.62384 | 1.83216 | 13.6046 | 22.7675 | 0.674203 |

© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Chamoso, P.; Rodríguez, S.; García-Ortiz, L.; Corchado, J.M.
Information Extraction from Retinal Images with Agent-Based Technology. *Processes* **2018**, *6*, 254.
https://doi.org/10.3390/pr6120254

**AMA Style**

Chamoso P, Rodríguez S, García-Ortiz L, Corchado JM.
Information Extraction from Retinal Images with Agent-Based Technology. *Processes*. 2018; 6(12):254.
https://doi.org/10.3390/pr6120254

**Chicago/Turabian Style**

Chamoso, Pablo, Sara Rodríguez, Luis García-Ortiz, and Juan Manuel Corchado.
2018. "Information Extraction from Retinal Images with Agent-Based Technology" *Processes* 6, no. 12: 254.
https://doi.org/10.3390/pr6120254