Automatic Tortuosity Estimation of Nerve Fibers and Retinal Vessels in Ophthalmic Images

Abstract

The tortuosity changes of curvilinear anatomical organs such as nerve fibers or vessels have a close relationship with a number of diseases. Therefore, the automatic estimation and representation of the tortuosity is desired in medical image for such organs. In this paper, an automated framework for tortuosity estimation is proposed for corneal nerve and retinal vessel images. First, the weighted local phase tensor-based enhancement method is employed and the curvilinear structure is extracted from raw image. For each curvilinear structure with a different position and orientation, the curvature is measured by the exponential curvature estimation in the 3D space. Then, the tortuosity of an image is calculated as the weighted average of all the curvilinear structures. Our proposed framework has been evaluated on two corneal nerve fiber datasets and one retinal vessel dataset. Experiments on three curvilinear organ datasets demonstrate that our proposed tortuosity estimation method achieves a promising performance compared with other state-of-the-art methods in terms of accuracy and generality. In our nerve fiber dataset, the method achieved overall accuray of 0.820, and 0.734, 0.881 for sensitivity and specificity, respectively. The proposed method also achieved Spearman correlation scores 0.945 and 0.868 correlated with tortuosity grading ground truth for arteries and veins in the retinal vessel dataset. Furthermore, the manual labeled 403 corneal nerve fiber images with different levels of tortuosity, and all of them are also released for public access for further research.

1. Introduction

As one of the most significant biomarkers, tortuosity of anatomical curvilinear organ has a close relationship with human diseases in medical images. In recent clinical research [1,2], it is confirmed that some morphological changes of curvilinear structures, nerve fibers, retinal vessels, corneal, coronary, and optic nerve are important cues for diagnosis many diseases. The tortuosity measurement can be used for multiple imaging modalities. The tortuosity is an important way to evaluate architecture and morphology of vasculature, and can be used for skin diseases detection in optical coherence tomography angiography and photoacoustic imaging [3]. Higher densities and tortuosity geometries is also usually indicating the location of cancer-induced angiogenesis in contrast-enhance ultrasound imaging [4]. Human eye is a small organ and full of many curvilinear structures which include nerve fibers and vessels. Diagnosis of many eyes and systemic diseases [1,5,6] are usually tested by quantitative properties of such curvilinear structure tortuosity. How to measure the tortuosity and what is the best measurement metrics are major topics in their research.

More specifically, ophthalmologists usually assess diabetic optic neuropathy [2] and hypertensive retinopathy [7] with tortuosity of corneal nerve fibers and retinal vessels, respectively. Figure 1 illustrates examples of nerve fiber and vessels with three-level tortuosity in corneal confocal microscopy (CCM) and retinal fundus image. Clinically, the tortuosity of these curvilinear structure is usually graded into three to five levels, or as normal/abnormal by ophthalmologists based on their experience. Nerve fibers and retinal vessels tortuosity are divided into different levels by the naked eye of clinicians after observing the CCM and retinal fundus image. However, the assessment made by clinicians is usually subjective and time consuming. Different clinicians will get different levels for the same image, and the same doctor also may give inconsistent decisions for the same image at a different time [1]. Therefore, it is desired to propose an objective, repeatable, highly adaptable and quantitative method to assess the tortuosity level of the curvilinear structure.

So far, various automatic tortuosity measurement approaches [1,2,7] fall into five steps: first, original image usually is enhanced by filters in the vessel segmentation problems. The three most influential filters are the eigenvalue-based filter, the wavelet transform based filter and the local phase based filter. The typical eigenvalue-based filter proposed by Frangi et al. [8] is based on eigenvalues of the Hessian matrix. The isotropic undecimated wavelet transform has recently been used for vessel segmentation and showed good accuracy and computational efficiency [9]. Local phase is an important local feature that can measure structural information of an image. It has recently been shown that this information can be used to enhance vessels in a more precise way and produce promising segmentation results [10]. Second, a segmentation step is utilized to get the curvilinear structure image from original medical image [11,12,13]. Third, a centerline map is extracted from curvilinear structure image by a skeletonization method, and all the intersection points of the centerline are detected. Then, the centerline is broken into small centerline fragments according to intersection points. In some cases, curve fitting or parametric pre-processing is performed on the fragments for the breakpoints connection and next calculation. Finally, the tortuosity of each fragment is calculated by different metrics which are shown in

Table 1. Summary of tortuosity metrics.

	Tortuosity Metrics	Symbols
1	Tortuosity Density [7]	TD
2	Absolute Curvature [14]	AC
3	Squared Curvature [14]	SQC
4	Weighted Absolute Curvature [15]	WAC
5	Absolute Direction Angle Change [16]	AAC
6	Arc-Chord Length Ratio [17]	ACR
7	Chord Length [18]	CHD
8	Curve Length [18]	CUR
9	Directional Change of a Line [19]	DCI
10	Inflection Count Metric [20]	ICM
11	Mean Direction Angle Change  [21]	MAC
12	Slope Chain Coding [22]	SCC
13	Tortuosity Coefficient [23]	TC

.

However, methods of the curvilinear tortuosity measurement usually lack of universally accepted definition, or standard specification [1,2,7]. Several definitions have been proposed in the literature with an aim to automatically assess the tortuosity. Scarpa et al. [2] compute the tortuosity based on frequency information such as the number of phase changes in the curvature. Some methods assess the arc length and chord length for the tortuosity estimation. The arc length is the pixel length between two endpoints of the centerline fragment, while the chord length is the linear distance between endpoints. Grisan et al. [7] multiply the sum of each ACR by the inflection points number for tortuosity assessment. Bracher et al. [18] assess the tortuosity by ratio of the arcs length to chords length (ACR). Bullitt et al. [20] provides a 3-D tortuosity analysis of vessels and evaluate three ACR related tortuosity metrics for different types of abnormality detection. Other methods evaluate the tortuosity based on angle and curvature. Semdby et al. [24] measure tortuosity by total curvature and distance factor calculations for the femoral arteriograms. This measure is suitable for detection of centerline fragment tortuosity with high-frequency changes. Hart et al. [14] differentiate simply tortuous retinal vessels from non-tortuours ones using total curvature and total squared curvature of vessel centerlines. Goh et al. [16] measure the vessels curvature for diabetic-related disease detection by direction change and ACR. Some other methods utilize coding and mode technique to measure tortuosity. Bribiesca [25] encode the retinal vessels by slope chain code and then get the tortuosity. Annunziata et al. [1,26] take the advantage of combining results from different tortuosity metrics, and then leverage the multinomial regression method to select the best combination.

In addition, the tortuosity evaluation still faces some problems to be solved, including human subjective errors, data variations, and metric differences. An image may be labeled by a clinician as the high tortuosity level while other clinicians as mild tortuosity level [1]. The analysis of the curvilinear organs is also a time-consuming task, since most of the analysis softwares still relies on the ophthalmologists manual measurement. It is worth noticing that the lack of available data limits the application of deep learning methods to tortuosity grading. Recently, a research [27] found that the performance of end-to-end tortuosity classification of corneal nerve images with AlexNet and VGG-16 did not exceed random accuracy on the tested dataset. The performance of conventional tortuosity measurements sometimes will also be affected by data differences, image pre-processing, and segmentation of curve structures. In order to make the tortuosity measurement more robust, the method is also invariant to scaling, rotation, and translation [6] in the clinical scence. Our previous work [28] evaluated the tortuosity assessment on the Retinex-based nerve fiber enhancement image. Therefore, an automatic and accurate measurement method is needed to evaluate the curvature of the curve. To tackle these problems, this paper introduces a new reliable framework for tortuosity assessment, which can conquer the human subjective error and metric variations. In addition, compared with the previous work [28], we evaluate the tortuosity assessment both on the nerve fiber and retinal vessel image based on the local phase tensor enhancement in order to verify the robustness of the method. Most of the tortuosity metrics are evaluated quantitatively on the dataset. The effectiveness of four enhancement methods on the tortuosity assessment is also evaluated on the internal and public segmentation method, respectively. The contributions of this paper are described as follows:

• We propose an automatic curvilinear tortuosity measurement method with the exponential curvature estimation, which measures the tortuosity value from the original image directly with few complex pre-processing steps of the traditional methods.
• Our proposed method is robust, which has been validated quantitatively using one retinal blood vessel tortuosity dataset and two corneal nerve tortuosity datasets. As a complementary output, we have made all the tortuosity datasets available online.

2. Methods

Our proposed framework consists of two stages: the curvilinear structure enhancement by the local phase tensor filtering, and the tortuosity estimation by exponential curvature calculation.

2.1. Curvilinear Structure Enhancement

Typically, the quality of acquired ophthalmic images exhibits varying conditions of illumination and noise, primarily due to the imperfect focus and acquisition process. The imbalanced intensity dims the appearance of curvilinear structures, such as nerve fibers in CCM imagery, and thus creates difficulty in distinguishing the nerve fibers and estimating their tortuosity from the background [29]. In this paper, we introduce the local phase tensor-based filter to enhance the curvilinear structures in ophthalmic images. The local phase is a measure of the structural information of an image, which is able to distinguish intrinsic features from background with the illumination invariant advantage [30]. Given a curvilinear structure image signal as $f\left(x\right)$, the definition of the local phase is:

$$\phi \left(x\right)=\arctan \left(\frac{E\left(x\right)}{O\left(x\right)}\right),$$

(1)

where $E\left(x\right)$ is the convolution operation response of $f\left(x\right)$ through the even filter. $O\left(x\right)$ is the response of convolution operation of $f\left(x\right)$ through the odd filter. Here, the odd filter is the Hilbert transform of the even filter. In practice, $E\left(x\right)$ and $O\left(x\right)$ are usually calculated by the log-Gabor filter:

$$E\left(x\right)=Re\left\{{\mathcal{F}}^{-1}(\mathrm{LG}\left(\omega \right)\times \mathcal{F}\left(x\right))\right\},$$

(2)

$$O\left(x\right)=Im\left\{{\mathcal{F}}^{-1}(\mathrm{LG}\left(\omega \right)\times \mathcal{F}\left(x\right))\right\},$$

(3)

where $\mathcal{F}\left(x\right)$ is the Fourier transform of the image signal. The $\mathrm{LG}$ indicate the log-Gabor filtering. ${\mathcal{F}}^{-1}$ is the inverse Fourier transform. Re and Im indicate the real part and the imaginary part of it expression, respectively. The symmetric aspect $T_{even}$ of local phase tensor is calculated as [31]:

$$T_{even}=\left[H\left(E\left(x\right)\right)\right]·{\left[H\left(E\left(x\right)\right)\right]}^T,$$

(4)

while the asymmetric aspect $T_{odd}$ is

$$\begin{array}{c}\hfill T_{odd}=-0.5\times \{[\nabla \left(O\left(x\right)\right)]·{[\nabla {\nabla }^2\left(O\left(x\right)\right)]}^T+[\nabla {\nabla }^2\left(O\left(x\right)\right)]·{[\nabla \left(O\left(x\right)\right)]}^T\},\end{array}$$

(5)

In these equations, ∇ and ${\nabla }^2$ denote the Gradient and Laplacian operations, respectively. H is the Hessian operation. The $T_{even}$ is the product of the term $H\left(E\right(x\left)\right)$ with its transpose. According to the conclusion in [30], the local phase tensor equation is:

$$\mathcal{T}=\sum _{\theta \in \Theta }\left[\sqrt{{\left(T_{even}^{\theta }\right)}^2+{\left(T_{odd}^{\theta }\right)}^2}\right]·\cos \left(\phi \right).$$

(6)

Furthermore, we also consider multiple orientations to make the rotation invariant tensor, where $\Theta =\{\frac{\pi }{16},\frac{2\pi }{16},\frac{3\pi }{16},\cdots ,\pi \}$. $\phi$ is the phase parameter, which is calculated from the local contrast. Figure 2a,e are the original images. Figure 2b,f illustrate the curvilinear structure enhanced images. Figure 2d,h are the curvature maps. Since the filters are usually calculated for the whole image, the tortuosity values of the bifurcation points are also calculated. However, the value of the bifurcation should not have impact on the evaluation of tortuosity, so we break the bifurcations and remove the tortuosity values.

2.2. Curvilinear Structure Tortuosity Representation

To precisely describe geometric variations of curvilinear structure, we assess the tortuosity by the exponential curvature estimation.

2.2.1. Representation of the Curvature Orientation

The curvature estimation is calculated from the parameters with position and orientation. Therefore we define a 3D space ${\mathbb{R}}^2⋊S^1$, in which every 2D curvilinear can be represented as a group $g=(\mathbf{x},\theta )$. The term $\mathbf{x}=(x,y)\in {\mathbb{R}}^2$ is identified by planar translations while $\theta \in S^1$ is the rotation. In this space, the image processing operations rely on a rotating frame of reference, such that

$$\{{\mathbf{e}}_{\xi }{|}_g,{\mathbf{e}}_{\eta }{|}_g,{\mathbf{e}}_{\theta }{|}_g\}=\{cos\theta {\mathbf{e}}_x+sin\theta {\mathbf{e}}_y,cos\theta {\mathbf{e}}_y-sin\theta {\mathbf{e}}_x,{\mathbf{e}}_{\theta }\}.$$

(7)

In order to describe the estimation process more clearly, we illustrate the orientation projection process in Figure 3. Take the synthetic image in Figure 3a as an example of the original curvilinear structures in 2D image. First the original curvilinear structures is deal with our enhancement method, and then projected to the 3D position-orientation space to get the orientation scores) [32,33] (Figure 3b). For each curvilinear, the mapping process is achieved through the wavelet transform and convolution operations. The transformation formula with continuous form is as follows:

$$S_f(\mathbf{x},\theta )={\int }_{{\mathbb{R}}^2}\overline{\psi \left({\mathbf{R}}_{\theta }^{-1}\left({\mathbf{x}}^{{}^{\prime }}-\mathbf{x}\right)\right)}f\left({\mathbf{x}}^{{}^{\prime }}\right)d{\mathbf{x}}^{{}^{\prime }},$$

(8)

where $S_f$ is the orientation score. In an image f, each spatial location $(x,y)$ of a curvilinear can be represented as a group $(\mathbf{x},\theta )$ in 3D space. Here, the $\psi \in {\mathbb{L}}_2\left({\mathbb{R}}^2\right)$ is the cake-wavelet transform, which is $\theta$-axis aligned with rotational symmetry. Finally, the curvilinear in 3D space is represented by the orientation scores for further curvature estimation.

2.2.2. Exponential Curvature Estimation

Intuitively, as shown in Figure 3b, the curvature of a curvilinear can be regarded as the rate of angular change of a moving point along the direction of the curve. In ${\mathbb{R}}^2⋊S^1$ 3D space, an exponential curve ${\gamma }_{\mathbf{c}}$ with arc length t is represented by tangent vector components and the rotation-invariant basis: i.e., ${\dot{\gamma }}_{\mathbf{c}}\left(t\right)=c^{\xi }{\mathbf{e}}_{\xi }{|}_{{\gamma }_{\mathbf{c}}\left(t\right)}+c^{\eta }{\mathbf{e}}_{\eta }{{|}_{{\gamma }_{\mathbf{c}}\left(t\right)}+c^{\theta }{\mathbf{e}}_{\theta }|}_{{\gamma }_{\mathbf{c}}\left(t\right)}$. Here the tangent vector is $\mathbf{c}={(c^{\xi },c^{\eta },c^{\theta })}^T$, while the basis is noted as $\{{\mathbf{e}}_{\xi },{\mathbf{e}}_{\eta },{\mathbf{e}}_{\theta }\}$. In orientation scores, exponential curves are presented as circular spirals, which are similar to straight lines with respect to the curved geometry in ${\mathbb{R}}^2⋊S^1$. With the spatial projections, each pixel on the curvature can be determined. For an exponential curve ${\gamma }_{\mathbf{c}}$ with the tangent vector of c, we note ${\mathbb{P}}_{{\mathbb{R}}^2}{\gamma }_{\mathbf{c}}$ as the projected curve. Therefore, the curvature $\kappa (\mathbf{x},\theta )$ is calculated by the term $\frac{c^{\theta }sign\left(c^{\xi }\right)}{\sqrt{|c^{\xi }{|}^2+{\left|c^{\eta }\right|}^2}}$.

Accurate local curvature estimation is achieved by fitting the exponential curve to the local curvilinear structure with the best alignment. The optimal tangent vector ${\mathbf{c}}^{*}$ of the best exponential curve fit is used to compute the local curvature $\kappa$, as shown in Figure 3b. We can solve the ${\mathbf{c}}^{*}$ by calculating the eigenvectors of the Hessian matrix. Here we take the Hessian matrix with $\mu$—normalized form previous orientation scores. Its mathematical expression is: ${\mathcal{H}}_{\mu }{S}_f=M_{{\mu }^{-1}}{\left(\mathcal{H}{S}_f\right)}^T{M}_{{\mu }^{-2}}\left(\mathcal{H}{S}_f\right)M_{{\mu }^{-1}}$, with

$$\mathcal{H}{S}_f=\left(\begin{array}{ccc}{\partial }_{\xi }^2{S}_f& {\partial }_{\xi }{\partial }_{\eta }{S}_f& {\partial }_{\theta }{\partial }_{\xi }{S}_f\\ {\partial }_{\xi }{\partial }_{\eta }{S}_f& {\partial }_{\eta }^2{S}_f& {\partial }_{\theta }{\partial }_{\eta }{S}_f\\ {\partial }_{\xi }{\partial }_{\theta }{S}_f& {\partial }_{\eta }{\partial }_{\theta }{S}_f& {\partial }_{\theta }^2{S}_f\end{array}\right),$$

(9)

For each element in the matrix, its first and second order derivative are calculated by rotation-invariant partial derivative [34]. Here, the derivative of position group is $\{{\partial }_{\xi },{\partial }_{\eta }\}=\{cos\theta {\partial }_x+sin\theta {\partial }_y,cos\theta {\partial }_y-sin\theta {\partial }_x\}$. The orientation element is ${\partial }_{\theta }$. $M_{{\mu }^{-1}}$ is the diagonal matrix [11].

From previous steps, we extract curvilinear structures from original images with the proposed enhancement method, and generate high-quality maps for the further calculations. Then, we obtained the orientation scores and curvature of local curvilinear structures. In Figure 3, we show this process with curvilinear structures with a different position and curvature. From the results of orientation and curvature of individual curvilinear fragment, we will evaluate the tortuosity all over the map. For tortuosity metric calculation, we consider the length, curvature of each curve and the proportion of different curvature in the map. The equation of the tortuosity metric is

$${\kappa }_{\exp }=\frac{1}{\mathcal{V}}{\int }_{-\infty }^{\infty }\left|{\kappa }_{{\mathbf{c}}^{*}}(\mathbf{x},\Theta \left(\mathbf{x}\right))\right|\mathcal{T}\left(\mathbf{x}\right)\mathrm{d}\mathbf{x},$$

(10)

where the ${\kappa }_{\exp }$ is our tortuosity metric, which is the normalized global absolute curvatures of the enhanced curvilinear. For an enhanced curvilinear structure map $\mathcal{T}\left(\mathbf{x}\right)$, the ${\kappa }_{\exp }$ is the sum of them with curvature weights $|{\kappa }_{{\mathbf{c}}^{*}}(\mathbf{x},\Theta \left(\mathbf{x}\right))|$. The $\mathcal{V}={\int }_{-\infty }^{\infty }\mathcal{T}\left(\mathbf{x}\right)d\mathbf{x}$ is the normalization factor which is the total sum of the $\mathcal{T}\left(\mathbf{x}\right)$. For the absolute curvature weight, the ${\kappa }_{{\mathbf{c}}^{*}}$ is a previous local curvature.

The $\Theta \left(\mathbf{x}\right)=\underset{{\theta }_i\in \frac{\pi }{\mathit{N}}\{1,\dots ,\mathit{N}\}}{arg\; max}\left\{S(\mathbf{x},{\theta }_i)\right\}$ selects the maximum orientation response angle as weight. $S(\mathbf{x},\theta )$ is our previous orientation scores. The weighted tortuosity metric can describe the curvilinear structure better. For a vessel or a nerve fiber image with multiple curvilinear structures, the weighted tortuosity metric can describe the curvilinear structure better, since many of the curvilinear structures are relatively smooth, while only a few of them are with high tortuosity. In order to capture the local curvature changes, we give large weight value for high tortuosity by $\Theta \left(\mathbf{x}\right)$ function while low weight value for low tortuosity. Then we take weighted sum rather than average for the whole curvilineear structure map.

3. Datasets and Metrics

In this paper, we use two corneal nerve datasets and one retinal vessel dataset for tortuosity evaluation, which are named CCM-A, CCM-B and RET-TORT. In order to validate the proposed tortuosity measure, the segmentation are manually done for all datasets. Particularly for the corneal nerve datasets (CCM-A and CCM-B), the reference fiber centerlines were manually annotated by ophthalmologists. For the retinal vessel dataset, the manually drawn vessel center lines are provided with the original dataset. For all the tested datasets, the centerlines of segmentation results are extracted and the intersection points on these centerlines are detected. When calculating the tortuosity degrees, the centerlines and are further broken into single curvilinear segments at each intersection point.

CCM-A contains 403 CCM images. In CCM-A, two clinicians independently graded the images into four levels according to the tortuosity from straight to heavy. The dataset is annotated in accordance with previously published protocol [35]. We have made this dataset available online (https://imed.nimte.ac.cn/corn.html). Level-1: 54 nerve fibers images with almost straight. Level-2: 212 images with mild tortuous. Level-3: 108 images are quite tortuous. Level-4: 29 images show heavy tortuous. Figure 4 illustrates nerve fibers images with four tortuosity levels, and their manually labeled centerlines.

CCM-B is a publicly-available dataset (http://bioimlab.dei.unipd.it/), with clinical annotations for corneal nerve fibers. The CCM-B database is composed of 30 CCM images. An expert grouped these images into three classes, based on degree of fiber tortuosity: low, mid, and high.

RET-TORT is the same publicly-available dataset with CCM-B. There are 60 total fundus images in this dataset. 30 arteries and 30 veins are extracted from them. All these images were examined by a retinal specialist, Names of these images are ordered by increasing tortuosity. For each image, the manually drawn vessel centerline is provided with the original dataset. As shown in Figure 5a–c are original images with low, middle and high tortuosity in RET-TORT. In this dataset, only the most important vessel was selected for tortuosity measurement and manual labeled. We overlaid the manual centerlines ground truth on the original images with yellow color.

For each tortuosity level, we compare the classification results with the ground truth by sensitivity (Sen), specificity (Spec), and accuracy (Acc) metrics. In order to evaluate tortuosity more accurately for different levels, as suggested in [1], we also take the weighted sensitivity (wSen), weighted specificity (wSpec), and weighted accuracy (wAcc) as metrics to demonstrate the overall performance. These metrics are defined as:

$$\mathrm{wSen}=\sum _{i=1}^L{r}_i\frac{{\mathrm{TP}}_i}{{\mathrm{TP}}_i+{\mathrm{FN}}_i},$$

(11)

$$\mathrm{wSpec}=\sum _{i=1}^L{r}_i\frac{{\mathrm{TN}}_i}{{\mathrm{TN}}_i+{\mathrm{FP}}_i},$$

(12)

$$\mathrm{wAcc}=\sum _{i=1}^L{r}_i\frac{{\mathrm{TP}}_i+{\mathrm{TN}}_i}{{\mathrm{TP}}_i+{\mathrm{TN}}_i+{\mathrm{FP}}_i+{\mathrm{FN}}_i},$$

(13)

let L be the total number of levels of tortuosity, for each i level tortuosity, $r_i$ is the percentage of level i images over all images. ${\mathrm{TP}}_i$ is the number of correctly classified images which blong to the i level. ${\mathrm{FN}}_i$ is the number of the i level images which are are falsely classified as other levels. ${\mathrm{FP}}_i$ is the number of other level images which are are falsely classified as i levels. ${\mathrm{TN}}_i$ is the number of other level images which are correctly classified.

4. Experimental Results

In experiment section, our proposed tortuosity estimation method is evaluated with combination experiments on these three tortuosity datasets. In addition, our proposed curvilinear structure enhancement method is evaluated by tortuosity grading and fiber segmentation. Furthermore, the tortuosity metrics in Table 1 are also evaluated quantitatively on the retinal vessel clinically.

4.1. Tortuosity Classification

First of all, we need to grade each image into different tortuosity levels. However, the number, length and curvature of curvilinear of each image are different, so it is a key problem to evaluate different tortuosity characteristics effectively [1]. As previous section stated, we should consider more about the measurement of a small part of the high tortuosity segments. Therefore, the tortuosity measurement for a whole image is calculated by the weighted average:

$$M=\frac{{\sum }_{i=1}^N{l}_i\times m_i}{{\sum }_{i=1}^N{l}_i},$$

(14)

where $l_i$ is the length of the ith curvilinear structure. N is the number of segments in a curvilinear structure map. For each ith curvilinear structure with length $\left(l\right)$, the $m_i$ is the tortuosity metric score. The tortuosity metric can be any one of the metrics in Table 1 as well as our proposed Kexp.

Here, we use 14 tortuosity metrics for experiment. The 14 features are the values of tortuosity metrics, which are listed in Table 1. First, we extract each image feature as a 14 dimensional vector, then the tortuosity classification is by the Support Vector Machine (SVM) with image features. For instance, as four-level grading is used to measure tortuosity, the one versus one strategy used by SVM for multi-class classification leads to six SVM classifiers, each of which is trained to discriminate one class from another.

SVM is a well-established classification technique and has been proved to be able to produce satisfactory results on many applications, in particular when the training data is not big enough as required by deep learning. We only have 400 images in this study, from our experience, this would be too small for deep learning models, and SVM is well suitable in the problem at this scale. In the experiment, we leverage the SVM in the Matlab, and all SVM parameters are set to default values. The parameters may not be the optimal but will provide a benchmark for future studies (The goal of classification here is just to verify the validity of this new method).

4.2. Nerve Fibers Tortuosity Grading

As shown in

Table 2. Comparison results of tortuosity classification on CCM-A and CCM-B.

		CCM-A Dataset					CCM-B Dataset
		Level-1	Level-2	Level-3	Level-4	Overall	Low	Mid	High	Overall
Annunziata [1]	Sen	0.718	0.644	0.660	0.707	0.663	0.783	0.743	0.761	0.766
	Spec	0.867	0.790	0.783	0.857	0.806	0.904	0.860	0.864	0.912
	Acc	0.858	0.780	0.759	0.843	0.790	0.877	0.826	0.847	0.848
Kexp based SVM	Sen	0.731	0.654	0.669	0.713	0.675	0.811	0.758	0.786	0.784
	Spec	0.878	0.802	0.794	0.865	0.816	0.914	0.865	0.887	0.921
	Acc	0.865	0.788	0.775	0.853	0.797	0.909	0.833	0.870	0.868
Kexp+ based SVM	Sen	0.742	0.674	0.683	0.734	0.715	0.824	0.765	0.790	0.810
	Spec	0.898	0.819	0.811	0.881	0.852	0.929	0.873	0.891	0.929
	Acc	0.881	0.799	0.793	0.864	0.820	0.914	0.841	0.883	0.882

, in this work, we calculate the tortuosity classification results for each level and the overall tortuosity level on both fiber dataset. For both the in-house CCM-A and public CCM-B datasets, the segmentation is manually done by ophthalmologists. In order to make a comparative study, we compared three methods: Annanziata [1], Kexp and Kexp+. The Annunziata method calculates ACR, TD, SCC, AC and WAC in Table 1 respectively, and get several features for an image. Finally, several features are fused by weighted average strategy, and the tortuosity of the image is classified into different levels by Multinomial Logistic Ordinal Regression. Kexp is our approach, but it does not include enhancement step. Kexp+ is our proposed method, which includes enhancement step.

In Table 2, the comparison results of tortuosity classification show that our proposed Kexp+ on enhanced images show better performance than the Annunziata’s both on CCM-A and CCM-B datasets. The evaluation metrics are wSen, wSpec, and wAcc scores. For the overall result, the kexp+ achieved the wAcc with 0.820 and 0.882 on the two datasets respectively, while the Annunziata’s approach are 0.790 and 0.848. The Kexp+ also outperforms the better classification results with regard to the wSen and wSpec in overall. In this experiment, we also further compare the reuslts level-by-level according to the tortuosity. Among different levels, we found that the level-1 column, level-4 column in CCM-A dataset and the low level column in CCM-B have the good classification results. The more levels are divided, the worse the classification results are. For Annunziata’s method, the wAccs of level-1, level-4 and low level are 0.858, 0.843 and 0.877, while the results of Kexp+ are 0.881, 0.864, and 0.914. In the group of low-level in CCM-B, two images were wrongly classified. Because of the interference of pathological features, some nerve fibers could not be traced completely.

4.3. Retinal Vessels Tortuosity Grading

In order to evaluate the tortuosity measurement performance of the proposed method on the RET-TORT dataset, we carried out a two steps experiments: In the first experiment, we use the manual annotation of the RET-TORT in order to show the effectiveness of the tortuosity metrics without accounting for possible mistakes of the segmentation. For each tortuosity metric, we calculated the tortuosity for arteries and veins images, respectively. Then we get the Spearman Correlation score of each metric correlated with tortuosity grading ground truth. The higher the correlation score is, the more accurate the metric is for the vessels tortuosity measurement. From the manual segmentation column of

Table 3. Spearman Correlation scores of tortuosity metrics for retinal vessels with manual and automated segmentation.

	Manual Segmentation		Automated Segmentation		Error (%)
	Arteries	Veins	Arteries	Veins
WAC	0.919	0.814	0.877	0.768	8.8
DCI	0.787	0.589	0.734	0.621	8.4
TC	0.949	0.853	0.919	0.812	7.1
CHD	0.801	0.662	0.756	0.638	6.9
AC	0.922	0.837	0.893	0.801	6.5
ICM	0.684	0.575	0.661	0.542	5.6
CUR	0.813	0.701	0.784	0.677	5.3
SCC	0.850	0.770	0.827	0.745	4.8
SQC	0.925	0.826	0.901	0.812	3.8
MAC	0.820	0.814	0.801	0.795	3.8
ACR	0.792	0.656	0.812	0.629	3.4
TD	0.890	0.760	0.912	0.753	2.9
AAC	0.838	0.695	0.841	0.677	2.1
Kexp	0.945	0.868	0.928	0.857	1.9

$\mathrm{Error}=\parallel \mathrm{avg}\left({\mathrm{SC}}_{\mathrm{manual}}\right)-\mathrm{avg}\left({\mathrm{SC}}_{\mathrm{automated}}\right)\parallel \times 100\%$.

, we found that the TC get the best score for arteries tortuosity while our Kexp is the best for veins. But some tortuosity metrics (e.g., DCI, ICM, ACR) have low correlations and lead some errors.

In the second experiment, we further validated our proposed method with the automated segmentation result on the RET-TORT dataset. We calculated Spearman Correlation scores of the tortuosity metrics on the segmentation results and compare their performances. The automatic segmentation pipeline is illustrated as Figure 5, we first perform the local phase filters enhancement on the original vessel images (Figure 5d–f). Then we segment the vessels by the Infinite Perimeter Active Contour with Hybrid Region (IPACHR) model (Figure 5g–i). After that, we get the skeletonization results of the segmentation and keep the most important vessel centerline for further tortuosity measurement (Figure 5j–l). As shown in the automated segmentation column of Table 3, all the tortuosity metrics are calculated based on the centerline, and the Spearman Correlation scores of each metric are computed correlated with tortuosity grading ground truth. Our Kexp is also computed by using our exponential curvature metric on the centerline. From the results, we found that our Kexp achieved the best Spearman scores on the automatic segmentation group. In addition, when comparing results between the manual segmentation and the an automated segmentation group, the Spearman scores of automated group is lower than mannual ones. Because the IPACHR segmentation model lead some segmentation errors. The automated segmentation results of our Kexp estimation process is also slightly lower than manual ones with 1.9% difference ratio. Therefore, our proposed tortuosity estimation method is more accurate and robust than other metrics.

4.4. Clinical Evaluation

In this part, we test the ability of our proposed method to distinguish health and pathologies in clinical practice. The dataset we used is CCM-A which contain four pathological conditions: patients with both diabetes and dry eye (23 patients, total 117 images), only with dry eye (28 patients, total 124 images), only with diabetes (24 patients, total 120 images) and healthy (28 patients, total 123 images). The number of subjects and images are shown in

Table 4. Different metrics in control subjects and other diseases cases.

Metrics	Healthy	Dry Eye	Diabetes	Dry Eye and Diabetes
AC	5.73 ± 4.89	5.56 ± 3.43	4.12 ± 2.23	4.72 ± 3.69
WAC ($\times {10}^3$)	0.47 ± 0.18	0.48 ± 0.17	0.50 ± 0.20	0.52 ± 0.26
AAC	14.93 ± 10.18	12.74 ± 7.95	13.51 ± 8.23	13.36 ± 9.75
ACR	0.98 ± 0.01	0.97 ± 0.01	0.97 ± 0.01	0.97 ± 0.01
CHD	73.31 ± 20.63	69.16 ± 19.06	62.02 ± 16.80	63.74 ± 21.72
CUR	74.85 ± 21.03	70.92 ± 19.48	64.11 ± 17.09	65.47 ± 21.63
DCI ($\times {10}^4$)	0.33 ± 0.26	0.37 ± 0.35	0.44 ± 0.42	0.42 ± 0.32
ICM	5.82 ± 1.64	5.60 ± 1.60	5.13 ± 1.50	5.23 ± 1.68
MAC	0.64 ± 4.16	0.80 ± 3.29	0.05 ± 3.67	0.36 ± 3.90
SQC ($\times {10}^3$)	0.30 ± 0.14	0.32 ± 0.14	0.36 ± 0.15	0.36 ± 0.17
SCC ($\times 10$)	0.36 ± 0.09	0.36 ± 0.07	0.35 ± 0.08	0.34 ± 0.07
TC ($\times {10}^2$)	0.42 ± 0.22	0.41 ± 0.13	0.43 ± 0.17	0.45 ± 0.22
TD	0.09 ± 0.03	0.10 ± 0.01	0.10 ± 0.01	0.10 ± 0.01
Kexp	0.17 ± 0.06	0.21 ± 0.04	0.24 ± 0.07	0.25 ± 0.05

. From the fourth column, we can see that eye disease will cause changes in curvature. Ppatients both with diabetes and dry eye have the highest Kexp score while the healthy group is the lowest. The Kexp score in the healthy group is lower than other eye diseases with p < 0.01. It means that our exponential curvature measurement is potentially able to distinguish the healthy from the disease groups well.

4.5. The Effectiveness of Curvilinear Structure Enhancement for Tortuosity Grading

Figure 6 shows intuitively why the proposed enhancement approach can improve the tortuosity estimation. Where Figure 6a is the raw image, Figure 6b,c are the 3D image after exponential curvature estimation and the 2D curvature image map after projection, respectively. Similarly, Figure 6d is an enhanced image obtained by our curvature enhancement method from the original image. Figure 6e,f are the results of exponential curvature estimation based on the enhanced image. The traditional method of tortuosity metrics of the original image is to calculate on the pre-segmented image of the original image. In contrast, our proposed tortuosity estimation is based on the curvature map. The tortuous structure has a higher value in the curvature map, while the background pixel is close to zero. Compared with Figure 6c,f, we can see that Figure 6c is greatly affected by the background noise of the original image. Figure 6f has the clearer structure and background.

We also obtained the tortuosity grading results after removing the enhancement step by the proposed method, referred as Kexp in Table 2. It can be observed that the tortuosity grading results of Kexp+ are better than the Kexp one. The proposed enhancement method is helpful to boost the final results with 0.023 and 0.014 in accuracy on both two fiber datasets.

4.6. The Effectiveness of Curvilinear Structure Enhancement for Fiber Segmentation

In this experiment, the effectiveness of different enhancement methods on the tortuosity assessment is also evaluated on our group internal and public segmentation method, respectively. For the segmentation methods, IPACHR [13] is the traditional enhancement mehtod which is proposed by our group. U-Net [36] is a popular deep learning based segmentation method which is widely used in medical image segmentation. For the enhancement methods, We compare the Weighted Symmetry Filter (WSF) [13], the Frangi proposed multi-scale vesselness filtering [8], and the proposed curvilinear structure enhancement method for fiber segmentation.

In order to evaluate different enhancement methods for different segmentation scenario, two segmentation methods are compared: one is the previous IPACHR method, another is the U-Net. In medical image processing domain, U-Net is a deep convolutional neural network for segmentation task, which classify every pixel of a medical image into different semantic label. Here, we verify the U-Net for nerve fibers segmentation with and without our enhancement methods separately, and the IPACHR model is also evaluated in this two cases. Because the U-Net is a trainable model, we randomly select $80\%$ images as the training set in CCM-A, and the left $20\%$ as the testing set. The IPACHR model is also validated on the same testing dataset.

We compare different enhancement methods from two evaluation criterias: the false discovery rate (FDR) and the sensitivity (Sen). The calculation of these two evaluation criteria is obtained by comparing the centerline predicted by the segmentation model with the ground truth centerline. Because the segmentation method only extracts one pixel wide centerline, the pixels of the extracted centerline are considered as true positive as long as it is within the range of three pixels around the ground truth centerline.

The segmentation performance is illustrated in

Table 5. Segmentation results of different enhancement methods with IPACHR and U-Net on nerve fiber images.

Enhancement	IPACHR		U-Net
	FDR	Sen	FDR	Sen
Raw	0.394 ± 0.007	0.738 ± 0.010	0.428 ± 0.008	0.749 ± 0.013
Vesselness filtering [8]	0.375 ± 0.006	0.753 ± 0.004	0.367 ± 0.010	0.552 ± 0.019
WSF [13]	0.363 ± 0.007	0.759 ± 0.004	0.353 ± 0.009	0.788 ± 0.004
Ours	0.357 ± 0.010	0.760 ± 0.005	0.338 ± 0.007	0.807 ± 0.005

. Our proposed method improve the Sen and decrease the FDR in both IPACHR and U-Net segmentation methods. Compared with the segmentation results of the raw image without enhancement, our method increase about $0.022$ in Sen, and decrease about $0.027$ in FDR by the IPACHR. For U-Net segmentation model, the Sen increase $0.056$, and the FDR decrease $0.090$. Compared with the results of WSF, the proposed method also demonstrates its promising result.

The last two images of Figure 7 demonstrate the fiber segmentation results obtained by U-Net over the original and enhanced image, respectively. In the red circle regions, we notice that our enhanced method can improve the segmentation performance. When comparing with the results from the raw image, our method can detect the fibers with noise matter much better, and it’s also more sensitive to some small fibers.

5. Conclusions

In ophthalmic images, the tortuosity change of curvilinear organs is associated with a number of ocular diseases. The assessment of tortuosity plays a significant role for diagnosis. However, it still has two problems. First, there is no general evaluation standard. Second, most of the tortuosity grading methods are highly dependent on the results of image pre-processing. The poor results of pre-processing and the method of tortuosity calculation itself will lead errors.

We employed a local phase tensor-based filter to improve the visibility of curvilinear structure, and a new exponential curvature estimation based tortuosity metrics for curvilinear structures is also illustrated. The proposed method has been validated on confocal corneal microscopy and retinal fundus dataset. The comparison results of tortuosity estimation with three datasets demonstrate that the proposed method has superior performance over the conventional measures.