A Robust Method of Eye Torsion Measurement for Medical Applications

Abstract

The detection of eye torsion is an important element for diagnosis of balance disorders, although it is rarely available in existing eye tracking systems. A novel method is proposed in this paper to provide robust measurement of torsional eye movements. A numerical approach is presented to estimate the iris boundary only according to the gaze direction, so the segmentation of the iris is more robust against occlusions and ambiguities. The perspective distortion of the iris pattern at eccentric eye positions is also corrected, benefiting from the transformation relation that is established for the iris estimation. The angle of the eye torsion is next measured on the unrolled iris patterns via a TM (Template Matching) technique. The principle of the proposed method is validated and its robustness in practice is assessed. A very low mean FPR (False Positive Rate) is reported (i.e., 3.3%) in a gaze test when testing on five participants with very different eye morphologies. The present method always gave correct measurement on the iris patterns with simulated eye torsions and rarely provided mistaken detections in the absence of eye torsion in practical conditions. Therefore, it shows a good potential to be further applied in medical applications.

1. Introduction

1.1. Background and Motivation

Eye torsion is an involuntary eye movement that aims to stabilize the vision during a head movement (i.e., head tilt). It is defined as a rotation of the eye around the LoS (Light of Sight). The LoS is the line that connects the point of regard and the center of the entrance pupil and it is used to indicate the gaze direction [1].

Torsional eye movement is an important element for diagnosis of balance disorders. Head tilt test is a simple method to stimulate eye torsion, in which the subject is asked to tilt his/her head and keep gazing at zero position (i.e., looking straight forward). It aims to provoke a compensatory torsional eye movement in the inverse direction of the head motion to keep a more stable image on the retina. It is a physiological ocular reflex and should be present in every healthy subject. Otherwise, the subject may report unstable vision or dizziness, so further examinations will be needed.

Eye torsion is especially interesting for diagnosis of BPPV (Benign Paroxysmal Positional Vertigo), the most common problem among the vestibular syndromes. The patient’s head is placed into different positions to trigger nystagmus (

Table A1. Vocabulary.

Word	Definition
Nystagmus	Nystagmus is an involuntary eye movement, described by a quick eye movement in one direction (fast phase) followed by a slow eye movement in the opposite direction (slow phase).
Semicircular canals	Semicircular canals (anterior, posterior and lateral) are the parts of the inner ear. One can see them as a three-axis gyroscope that measures the angular acceleration of the head in any direction.
Scleral search coil	Scleral search coil is the gold standard method for 3D (horizontal, vertical and torsional) eye movement tracking. The subject wears a contact lens with integrated coils that are further detected within magnetic fields. The measured voltages are next processed to compute the 3D eye orientation.

) and vertigo during the examination. The presence of eye torsion will allow practitioners to distinguish if either the anterior or posterior semicircular canal (Table A1) is affected, because it only occurs in case of a posterior BPPV in addition to a high-speed vertical nystagmus. It is one of the most challenging medical applications, and none of the existing eye tracking systems is able to provide reliable eye torsion measurement in such conditions.

The measurement of eye torsion is rarely available in existing eye tracking systems due to its complexity in practice. A sub-degree precision in eye torsion measurement is generally reported on artificial and simulated eye images in related works [2,3], whereas it is difficult to maintain the reported precision in practice due to some inherent problems (e.g., occlusions, iris distortion, various morphology). Otero-Millan et al. compared their method of eye torsion measurement to the scleral search coil (Table A1) on human subjects, but they did not conclude on either the precision or robustness of their method [4].

The robustness of eye torsion measurement in practice is especially focused on in the present work. The goal is to provide robust torsional eye movement tracking with sufficient precision regarding the requirements of relevant medical applications.

1.2. Related Works

Eye torsion is detected in 2D eye images by measuring the angular shift of the iris pattern around the pupil center. It corresponds to measuring the rotation of the iris around the LoS in the 3D pupil-iris plane. Such an approach is commonly adopted in related works, because it requires neither invasive markers [5] nor additional illumination [6].

The boundaries of the pupil and iris are firstly detected in eye images. Several image-based methods are proposed in related works, such as binarization [7,8], edge detection [9,10], ray projection [11,12], etc. The detected candidate boundaries are generally refined to reduce the impact of noises (e.g., occlusions, mistaken points). The eyelids are detected and excluded in some related works to prevent occlusions [4,13]. The CRs (Corneal Reflections) are often filled to recover the connectivity of the detected pupil [14,15]. Next, the candidate boundary points are fitted to a regular shape (i.e., a circle or an ellipse) to provide smooth eye segmentation [4,8,12], i.e., segmentation of the pupil and iris.

In general, the detected iris is unrolled into a rectangular shape via polar transformation, so the linear displacement of a rectangular pattern is measured rather than the angular shift of an annular iris. Of course, the angular shift of the iris can be directly measured along a curved path. However, this will increase the complexity of the calculation (e.g., application of TM algorithm along a non-linear path) and the processing will be consequently slowed. For these reasons, the iris unroll is quite a mandatory step for eye torsion measurement as proposed in most related works [4,9,16].

Several techniques can be used to measure the angular shift of the iris pattern, such as cross-correlation, feature tracking and template matching. The displacement of the gray-level distribution along several samples (e.g., narrow arcs, arc-shaped windows) are measured in cross-correlation based systems [2,16]. The feature tracking method consists of measuring the displacements of some stable features of the iris over time [3,17]. However, both methods suffer from the occlusion and distortion of the iris in practice.

Template Matching (TM) is widely used in recent eye tracking systems to measure torsional eye movement [4,9]. A referent iris is firstly unrolled and defined as a unique texture, which is further compared with the current iris. This approach is more robust against occlusions in comparison with other methods, since a unique and global texture is considered rather than some pieces of iris (i.e., arc or window samples, individual features). Nevertheless, it is more sensitive to perspective distortion of the iris pattern at eccentric eye positions [4]. Therefore, it is important to rectify the iris before the measurement of eye torsion in a TM approach.

1.3. Paper Organisation

A robust method of eye torsion measurement is presented in this paper (Figure 1a). The iris is firstly detected at a referent eye position (i.e., zero gaze direction) using a traditional image-based approach. It is then unrolled into a rectangular pattern of fixed size. The location of the iris boundary is estimated in subsequent images by transforming the initial iris according to the current gaze direction (Figure 1b). This transformation is also used to map each pixel of current iris pattern onto the referent one. This allows the system to always unroll the iris in the same way and to correct the perspective distortion of the iris pattern at eccentric eye positions. The contrast of the unrolled iris is next enhanced and the noises (e.g., reflections, intruded eyelids) are removed. A TM algorithm is then applied on the unrolled iris pattern to measure the shift distance between its referent position to its current position (Figure 1c). The angle of the eye torsion is finally obtained by converting the measured distance from pixel to degree.

The robustness of present method in eye torsion measurement is assessed in a gaze test. The results are reported and discussed at the end of this paper, and some improvement ideas are given as well.

2. Method

A monochrome camera (DMK 22BUC03, The Imaging Source Europe GmbH, Bremen, Germany) is mounted on a head support for capturing front images of the eye at a distance of about 5 cm. Four NIR (Near Infrared) illuminators with a peak wavelength of 880 nm are used to provide invisible and constant illumination. A NIR bandpass filter is added to cut off the visible light sources. The camera is configured to perform an image acquisition at 30 FPS (Frames Per Second) with a resolution of $744\times 480$ pixels in grayscale.

The present method is coded in C++/CLI language with Visual Studio 2017 community IDE (Integrated Development Environment). The SDK (Software Development Kit) provided by the camera manufacturer is used to control the connection between the computer and the camera. The OpenCV library [18] is integrated in the current implementation. The image storing format “cv::Mat” is adopted to facilitate access of the image data and the use of some available functions.

2.1. Iris Segmentation

2.1.1. Detection of the Initial Iris

The pupil is firstly detected at the referent eye position (i.e., zero gaze position) in the image. It is represented by an ellipse, defined by:

$$\frac{x_{pup}^2}{a_{pup}^2}+\frac{y_{pup}^2}{b_{pup}^2}=1$$

(1)

where $a_{pup}$ and $b_{pup}$ are the semi-major and semi-minor radii of the ellipse, respectively, and $a_{pup}\ge b_{pup}>1$. An additional parameter, ${\theta }_{pup}$, is defined as well, which corresponds to the rotation angle of the current ellipse around its center.

The iris is assumed to be concentric with the pupil in the present work, since the pupil is the aperture located in the middle of the iris. The term “iris boundary” will refer to the outer boundary of the iris in this paper, because its inner boundary is actually the pupil’s contour. Therefore, the iris boundary is modeled as a scaled-up ellipse with respect to the pupil and it shares the same center with the latter. In other words, there is no need to locate all iris boundary points to fit an ellipse, only the ratio between the radii of the iris and the pupil is of interest.

Several rays are projected in radial directions, departing from the pupil center and passing through the points of the pupil boundary (Figure 2a). The range of projection is limited in [0°, 45°] and [135°, 180°] to avoid frequent iris occlusion by eyelids. The cumulative gray-level is calculated along each ray within a predefined searching range as shown in Figure 2c. More specially, the grayscales of the pixels that are located on each searching ray are memorized and are used to compute the curve of cumulative gray-level along the corresponding ray. The start points of the searching rays are located at a certain distance from the pupil contour to avoid the transition part between the pupil and the iris. A maximum searching distance is empirically defined as well, which is long enough to certainly include the iris boundary but does not go too far from the iris.

An example of the obtained cumulative gray-level curve is shown in Figure 2d. A simple method is used to locate the candidate iris boundary point $p{t}_{iris}$. This consists of finding the farthest point of the curve to the straight line that connects two end points of the curve. The ratio $\Vert cp{t}_{iris}\Vert /\Vert cp{t}_{pup}\Vert$ is then calculated and memorized for each ray. The edge of the lower eyelid or iris features may sometimes be selected rather than the iris boundary due to the small difference of the gray-level between the iris and its surrounding area. Outliers should thus be excluded, and it is done via an iterative approach. The average of the memorized ratios is firstly calculated at each loop. The candidate points having a ratio that is differed more than ±10% from the average are considered as outliers and are rejected. This is repeated until convergence of the average or if the maximum number of loops is reached, which is arbitrarily set to 3.

The initial iris boundary is then determined at the referent eye position by scaling up the ellipse that represents the pupil (Figure 2b):

$$a_{iris}=s·a_{pup}, b_{iris}=s·b_{pup}.$$

(2)

where $s$ is the mean ratio calculated after the outlier elimination. The center of the iris and its rotation angle remain the same as the pupil’s.

2.1.2. Estimation of Iris Boundary

Occlusion and ambiguity are two common and inherent problems in the detection of the iris boundary. The iris boundary is often occluded by eyelids, especially at eccentric eye positions. The grayscale of the iris sometimes does not differ a lot from its neighboring areas (i.e., pupil, sclera, eyelids) because of morphology variation between individuals, over or insufficient illumination, etc. A numerical method is thus proposed to overcome these problems, for which the location and shape of the iris boundary are estimated only according to the current gaze direction.

A simplified definition of 3D eye movement involves assuming a rotation of a spherical eyeball around three orthogonal axes that intersect at the center of the eyeball, i.e., the relative horizontal and vertical axes and the LoS. An eye coordinate frame $\left(O_{eye}, Xe-Ye-Ze\right)$ can thus be defined as shown in Figure 3. This definition has been used in many relevant fields, because it reduces both physiological and kinematic complexities of eye movement while preserving a sufficient accuracy in its analysis.

The 2D (horizontal and vertical) eye rotation can then be described by a $3\times 3$ rotation matrix, which is composed following the well-known Fick sequence [19]:

$$R_{H\&V}=R_V\left(\varphi \right)R_H\left(\theta \right)=\left[\begin{array}{ccc}\cos \theta & 0& \sin \theta \\ \sin \varphi \sin \theta & \cos \varphi & -\sin \varphi \cos \theta \\ -\cos \varphi \cos \theta & \sin \varphi & \cos \varphi \cos \theta \end{array}\right]$$

(3)

where $R_H$ and $R_V$ represent the horizontal and vertical eye rotations around the axes $Ye$ by $\theta$ and $Xe$ by $\varphi$, respectively. $R_{H\&V}$ is equal to the identity matrix $I_{3\times 3}$ at referent eye position.

Let $e_{ref}$ denote the initial iris detected at referent eye position, with its radii $a_{ref}$ and $b_{ref}$, and the rotation angle ${\theta }_{ref}$ around its center $p_{ref}$. The semi-major radius $a_{ref}$ of the iris is assumed to be invariant for each user during a same eye tracking session, because the iris size of each individual always remains constant, and the hardware configuration is kept unchanged during each eye tracking session.

Figure 4a illustrates the top view of the eye coordinate frame $\left(O_{eye}, Xe-Ye-Ze\right)$ in relation to the image plane and the corresponding eye image at referent position. The iris plane $P_{iris}$ is perpendicular to the LoS (i.e., the axis $Ze$), and it is assumed to be parallel with the image plane $P_{img}$ at referent position, i.e., $P_{img}$ is defined by axes $\left(Xe-Ye\right)$. Thus, $e_{ref}$ detected in the front image also defines the initial iris that lies in the plane $P_{iris}$.

Let $m_{ref}={\left[x_{ref},y_{ref}\right]}^T$ denote any boundary point of the initial iris $e_{ref}$ in the iris plane. $M_{ref}={\left[x_{ref},y_{ref},r_{eye}\right]}^T$ is the 3D coordinates of $m_{ref}$ in the eye coordinate frame. $r_{eye}$ is the radius of the spherical eyeball (i.e., 13.54 mm [20]) in pixel unit, which is converted using camera information.

The iris plane is rotated from the referent eye position by $R_{ref\to curr}=R_V\left(\varphi \right)R_H\left(\theta \right)$ as illustrated in Figure 4b. The new location of the 3D point $M_{ref}$ is denoted by $M_{curr}$, and it can be easily computed as:

$$M_{curr}=R{’}_{ref\to curr}·M_{ref}$$

(4)

The abscissa of the image plane is in fact flipped along the horizontal axis $Xe$, considering the position of the camera in relation to the eye coordinate frame. The horizontal rotation around the axis $Ye$ should thus be flipped as well, defining $R{’}_{ref\to curr}=R_V\left(\varphi \right)R_H\left(-\theta \right)$.

The viewed iris $e_{curr}$ in the current eye image is the projection of $e_{ref}$ lying in the rotated $P_{iris}$ on the image plane $P_{img}$. In other words, the boundary points $m_{curr}$ of $e_{curr}$ are the projection of $M_{curr}$ on the plane $\left(Xe-Ye\right)$. The following relation is hence established using (4).

$$m_{curr}=t_{shift}+{\left(R{’}_{ref\to curr}\right)}_{2\times 3}·M_{ref}$$

(5)

${\left(R{’}_{ref\to curr}\right)}_{2\times 3}$ contains the first two rows of $R{’}_{ref\to curr}$ for the sake of simplicity, because only the projection of $M_{ref}$ on the plane $\left(Xe-Ye\right)$ is of interest. $t_{shift}$ describes the translation between the initial iris center $p_{ref}$ and the current iris center $p_{curr}$ in the image plane, calculated by

$$t_{shift}=p_{curr}-{\left(R{’}_{ref\to curr}\right)}_{2\times 3}·{\tilde{p}}_{ref}$$

(6)

where $p_{curr}$ is the detected pupil center in the current eye image. ${\tilde{p}}_{ref}={\left[p_x,p_y,r_{eye}\right]}^T$ is the 3D coordinates of the initial iris center $p_{ref}$.

Hence, the iris boundary in the current eye image is estimated via relation (5). The parameters of current ellipse $e_{curr}$ are then determined using the available function “fitEllipse” of OpenCV library [18]. It takes a list of known boundary points, $m_{curr}$, and calculates the best-fit ellipse via a least-square approach. Such a function is chosen because the projection of the ellipse $e_{ref}$ lying in the rotated plane $P_{iris}$ on the image plane $P_{img}$ is still an ellipse (i.e., $e_{curr}$). The input points $m_{curr}$ thus perfectly fit an ellipse, and this function will offer a quick and accurate determination of $e_{curr}$.

2.2. Iris Unroll and Distortion Correction

The iris is unrolled into a rectangular shape to simplify the application of the measurement method (i.e., TM). The size of the rectangular iris pattern is kept unchanged for each user during a same eye tracking session. Its width is always set to 360 pixels, so a shift of 1 pixel along its abscissa will correspond to an eye torsion of 1°. Its height, $H$, is calculated as below using the major radii of the initial iris and pupil, $a_{ref}$ and $a_{pup}$, respectively.

$$H=a_{ref}-\left(a_{pup}+d\right)$$

(7)

The iris unroll is performed along the mapping axes $\gamma$ and $r$, corresponding to the angle of the eye torsion and the distance from the outer iris boundary (Figure 5a), respectively. The spacing distance, $d$, is used to avoid including the transition part between the iris and pupil. The height, $H$, is not fixed for every user to prevent artificial errors that may be added during the resizing of the iris pattern.

Let $m_{unroll}$ denote a pixel of the unrolled iris pattern located at $\left(\gamma , r\right)$, and the coordinates of its corresponding pixel $m$ in the eye image is computed as below:

$$\begin{gathered} m=p+\left[\begin{array}{c}a\cos \gamma \cos \theta -b\sin \gamma \sin \theta \\ a\cos \gamma \sin \theta +b\sin \gamma \cos \theta \end{array}\right], \\ \mathrm{with} a=a_{ref}-r \mathrm{and} b=b_{ref}-r, r\in \left[0,H-1\right]. \end{gathered}$$

(8)

where $p={\left[p_x,p_y\right]}^T$ and $\theta$ are the center and rotation angle of the detected pupil in the current eye image, respectively. $a_{ref}$ and $b_{ref}$ are the radii of the ellipse $e_{ref}$ that represents the initial iris.

The relation (8) is directly used for unrolling the initial iris $e_{ref}$ at the referent eye position. Otherwise, the corresponding point of $m$ is sought in the current eye image according to the gaze direction, via the relation (5), which was defined in the previous sub-section.

The iris distortion at the eccentric eye position is corrected as well using this approach. Since each pixel $m_{unroll}$ of the unrolled iris is always paired with the same point $m_{ref}$ of the referent iris, that is further mapped to a point $m_{curr}$ of the current iris $e_{curr}$. An example is shown in Figure 6. The iris pattern (b) is directly unrolled from (a) using the relation (8), whereas (c) is unrolled via the mapping relation (5). One can see that pattern (b) is gravely deformed, which is obviously not reliable for further processing. This deformation is corrected in pattern (c) using the present method, since it is constructed using the pixel-to-pixel mapping relation between the referent iris and current iris.

2.3. Eye Torsion Measurement

2.3.1. Pre-Processing of Iris Pattern

A mask containing the noises is created and filled with black pixels, including the upper eyelid, eyelashes and CRs. The masked pixels of the unrolled iris pattern will be further ignored when applying the TM algorithm.

The iris is often occluded by eyelids depending on the user’s morphology and gaze direction. The upper iris is mostly impacted in practice, because the upper eyelid comes to cover the eyeball during eye blink and closure, while the lower eyelid is almost immobile. In this respect, a mask is determined to exclude the top one-third of the pixels of the estimated iris before applying TM, where the occlusion frequently occurs (Figure 7a). This ratio is empirically defined and tested in a preliminary test. It almost includes the whole intruded eyelid area regardless of the user’s morphology. A rectangular mask is chosen rather than a fan-shaped mask (Figure 7b) to preserve a part of useful iris texture that often appears near the pupil boundary.

Eyelashes generally appear as dark lines in the eye image. They are excluded via a simple thresholding, i.e., all pixels darker than the pupil are rejected from the unrolled iris pattern. No more complex operation is considered in the present work for eyelash exclusion, considering the thickness of eyelashes and the frequency of the occlusion.

A two-stage method is proposed to exclude CRs from the unrolled iris pattern (Figure 8). The very bright pixels are firstly detected, corresponding to the central part of the CRs. Next, the border of the detected bright spots is extended outwards by 3 pixels in 8 radial directions. This aims to exclude the halos around the CRs, which often have a similar grayscale in comparison with the iris’.

The contrast of the iris pattern is enhanced via histogram equalization to highlight the characteristic iris features (Figure 9). It is necessary for the current TM algorithm, which often provides a mistaken measurement on raw iris patterns with low contrast.

The histogram equalization is only applied on the non-masked area of the iris pattern, where the pixels with the minimum and maximum grayscale are searched. They are next used to modify the image contrast as below:

$$f^{\prime }\left(\gamma ,r\right)=\frac{255}{max-min}\left(f\left(\gamma ,r\right)-min\right)$$

(9)

where $f\left(\gamma ,r\right)$ and $f’\left(\gamma ,r\right)$ are the initial and modified grayscale of the pixel located at $\left(\gamma ,r\right)$ in the unrolled iris pattern before and after histogram equalization, respectively.

2.3.2. Template Matching

A referent iris should be determined to be further compared with the current iris and thus to compute the angle of the eye torsion.

The initial iris $e_{ref}$ is detected at the referent eye position, and it is unrolled into a rectangular pattern. The referent iris pattern is then extended by 15° to avoid an abrupt change in the torsion measurement around 0° and 360°. The value 15° corresponds to the maximum eye torsion angle observed under a large head motion (i.e., ±40°) [21]. It is used to limit the searching range of the TM algorithm and thus to save processing time. In this respect, the pixels located within $\left[0°, 15°\right]$ and $\left[345°, 360°\right]$ are repeated at two ends of the unrolled iris as illustrated in Figure 10.

The available function “matchTemplate” of OpenCV library [22] is used to measure the 1D displacement of the unrolled iris pattern. The most similar part of the referent iris pattern to the current one is searched by sliding the latter along the abscissa. This corresponds to performing a search over the entire 360° using the circular redundancy defined above within a searching range of ±15°. The parameter “CV_TM_CCORR_NORMED” is used to calculate the correlation between the current iris and the sliding window in the referent iris, because it is more robust against contrast variation.

The output is then the one-dimensional displacement of the iris pattern from the referent position to its current position. It represents directly the angle of the eye torsion in degree, since the width of the unrolled iris pattern is fixed to 360 pixels (i.e., 1°/pixel).

The function “matchTemplate” is considered as a black box in the present work. It was already adopted by Otero-Milan et al. [4] for eye torsion measurement. Moreover, its performance on current pre-processed iris patterns was validated in a preliminary test. The referent iris patterns of five different subjects were manually shifted along the abscissa within $\left[-20°, 20°\right]$ by a step of 1°. The applied TM algorithm always gave the correct measurement on the pre-processed iris patterns within the searching range (i.e., ±15°) and returned a null result (i.e., zero torsion) otherwise. The same input iris patterns were also tested without contrast enhancement, and the applied TM algorithm usually misidentified the patterns with low contrast. Therefore, the application of some image enhancement (e.g., histogram equalization) is necessary for use of the current TM algorithm to guarantee its precision in eye torsion measurement.

The TM algorithm is applied on the entire unrolled iris pattern rather than on a part of it as proposed in related works [9] to reduce potential mismatching errors around the masked areas. Because the amount of useful pixels of the iris will vary depending on the partition size, some partitions will contain more masked pixels than others and may thus influence the matching result.

3. Results

The head-fixed prototype was initially designed for 2D (horizontal and vertical) eye movement tracking in a previous work, for which head movements are forbidden. However, it is obviously not suitable for stimulating eye torsion with head movements, so it is difficult to assess the performance of the proposed method with the present prototype. Moreover, there is no standard that clearly indicates the requirements of eye torsion measurement for medical applications. For these reasons, the authors have decided to firstly validate the principle of the present method and evaluate its robustness in a gaze test. The FPR (False Positive Rate) is measured for this purpose to verify if the system does not give mistaken results in the absence of eye torsion.

Five participants (2 males and 3 females) with very different eye morphologies were enrolled in the test, including 3 European eyes, 1 Asian eye and 1 African eye. They were asked to look at the target displayed on the monitor while keeping their head upright and steady. Only zero eye torsion can be measured in the current test, because head movement is forbidden and there is no visual stimulus that may trigger eye torsion in healthy subjects [23].

The experimental setup is illustrated in Figure 11. A monitor is placed at 50 cm in front of participant’s eye, such that the target displayed at zero gaze position is already fixed by the participant when the latter is looking straight forward. The targets are displayed in a predefined order. Only one target is shown at a time and it is displayed for 3 s at each position. The test was performed under controlled lights (i.e., low and constant illumination) and the background of the monitor is set to black color to avoid visual distraction. The proposed eye torsion measurement method was executed on a MacBook Pro with an Intel Core i7 (2.8 GHz, Dual-Core), running Microsoft Windows 8.1 under Boot Camp.

A set of 6174 front images of the right eyes were assessed in total. A visual inspection was performed before the analysis to remove the images with missing or inaccurate pupil segmentation in order to focus on eye torsion detection. No eye torsion was observed during the visual inspection, so all non-zero measurements are considered as mistaken detections. The measured FPR is reported in

Table 1. Measured FPR of eye torsion detection.

Participant ID	Number of Images with Mistaken Detection	Number of Input Images	FPR
Participant 1	8	1208	0.7%
Participant 2	9	1258	0.7%
Participant 3	0	1258	0.0%
Participant 4	77	1193	6.5%
Participant 5	112	1257	8.9%
Overall	206	6174	3.3%

.

The accuracy of the present method in the absence of iris distortion and occlusions was demonstrated using the manually shifted iris patterns, as described in Section 2.3.2.

A large part of the input eye images (i.e., about 2/3) were taken at eccentric eye positions in the current gaze test, and the morphologies of the participants are very different between each other. The obtained results somehow reflect the performance of the proposed methods of iris segmentation and iris rectification (i.e., correction of the iris distortion), showing a great robustness against occlusion and iris distortion in practical conditions, because a very low mean FPR of eye torsion detection is reported (i.e., 3.3%) and only an FPR less than 1% is reported for more than half of the five participants in the current test.

Therefore, the principle of the proposed method in eye torsion measurement is validated, because it always gave correct measurements on the iris patterns with simulated eye torsions (i.e., manually shifted patterns) and rarely provided mistaken detections in the absence of eye torsion in practical conditions.

The present eye torsion measurement method was executed on a single thread and only takes 13 ms per image on average. The image acquisition is set to 30 FPS in the current system, i.e., the interval time between two consecutive frames is about 33 ms. This frame rate was chosen because it is sufficient for most clinical tests that may require 3D (horizontal, vertical and torsional) eye movement tracking and it is commonly adopted by eye tracking systems that are available on the market for medical applications. The required computing time for pupil detection and gaze estimation is about 12 ms per image in the present system. Therefore, the proposed method of eye torsion measurement can be implemented in an online 3D eye tracking system at 30 FPS on a computer with ordinary settings (i.e., no specific hardware configuration is required).

4. Discussion

A wearable eye tracking system is clearly more adequate for eye torsion recording and measurement in practice as previously discussed. Therefore, it will be the first priority for the authors to implement the proposed method into a wearable eye tracking system, so one will be able to freely evaluate its performance (i.e., precision, robustness and computing time) in real application conditions.

The proposed iris segmentation method is more robust against occlusion and ambiguity than image-based approaches, because the iris boundary is computed according to the gaze direction, and the latter is estimated using the result of pupil segmentation, which is less sensitive to noises due to the smaller pupil size than the iris. Nevertheless, the performance of the proposed method still relies on the quality of 2D (horizontal and vertical) eye movement tracking, so some improvements are expected to improve its robustness and to make it useful for every eye tracking system despite the provided precision in gaze estimation. One of the ideas consists of checking additionally the consistency of the obtained result according to image information. More specially, it involves performing a quick detection of the iris boundary within a small range (e.g., a narrow annulus) around the estimated iris. It will allow the system to detect unreliable estimations of the iris boundary as well as the gaze direction and will not cause important additional processing time.

In order to identify the causes of higher FPR that were reported for the two last participants, the referent iris patterns of all participants were extracted and compared. They are listed in

Table 2. Referent iris patterns of five participants, sorted in descending order of the measured FPR of eye torsion detection.

Participant ID	FPR	Referent Iris Pattern
Participant 5	8.9%
Participant 4	6.5%
Participant 1	0.7%
Participant 2	0.7%
Participant 3	0%

and are sorted in descending order of the measured FPR.

One can see that both iris patterns in question contain a part of the lower eyelid, which seems to be the main source of error. A rectangular mask is currently used to prevent the occlusion of the upper eyelid, which can also be applied on the lower eyelid. However, it is more difficult to determine a fixed mask in this case, because the shape and intruded surface of lower eyelid vary a lot between individuals. Therefore, a new method has to be defined in future developments to accurately remove all intruded eyelids from the iris pattern.

The iris pattern of Participant 5 is more textureless and blurred than the others. A global and unique iris texture is considered in the present work, for which its uniqueness is determined by some distinct textures or characteristic features. The iris pattern of Participant 2 is a great example. Otherwise, the system will not be able to measure the displacement of a flat iris pattern, due to the absence of distinctive markers. Some advanced filters were applied in related works to bring out the iris texture [4], but their effectivity on such a textureless iris needs to be evaluated. The modification of the hardware will be another choice. The image resolution can be increased to capture more details of the iris, but the processing will be consequently slowed. One can also try to highlight the striations of the iris surface by adding some transverse light sources, while extra reflections will be observed in the captured eye images. All these propositions show some advantages and drawbacks, so an assessment on a larger image set taken from more participants will be needed to make a decision.

The width of the unrolled iris pattern is set to 360 pixels in the present work to offer a precision of 1° in the measurement of torsional eye movement. There is no available standard that clearly determines the clinical requirement of the precision in eye torsion measurement, and it is still unclear how practitioners interpret and use these results in diagnosis. For these reasons, the authors decided to only detect the presence of eye torsion with rough measurements in this work. A collaboration with practitioners is expected in future works to better determine their needs, and the precision of the present method in eye torsion measurement will be further improved according to the determined requirements.

Only five participants were enrolled in the current assessment. It was sufficient to validate the principle of the present method in eye torsion measurement as a preliminary test, whereas it will be important to perform a complete and statistical assessment to evaluate its performance (i.e., precision, robustness and processing time) with a large and significant sample size in future works. The precision can be assessed using artificial eyes or on the simulated eye images, and the robustness can be further evaluated through a clinical assessment. A comparison test with existing systems for eye torsion measurement will be interesting, especially with the gold standard (i.e., scleral search coil) that is already widely used in medical applications for 3D (horizontal, vertical and torsional) eye movement tracking. The proposed method may have to be speeded up to address the requirements of different clinical tests, so its processing time will be independently assessed for this reason.

5. Conclusions

The robustness of eye torsion measurement is often challenging in practice, so it is rarely proposed in current eye tracking systems that are available on the market. In this respect, a novel method is presented in this paper to provide robust detection of eye torsion for medical applications. Some inherent problems (i.e., occlusion and ambiguity) that were reported in related works for extraction of the iris are overcome using a numerical method. A TM approach is adopted to measure the angle of the eye torsion on the pre-processed and unrolled iris patterns, which is robust against noises and iris distortion. The principle of the proposed method is validated and its robustness in practice is assessed. It always gave correct measurements on the iris patterns with simulated eye torsions and rarely provided mistaken detections in the absence of eye torsion in practical conditions, i.e., a very low mean FPR (i.e., 3.3%) is reported in a gaze test on eye images of five different participants. Hence, the proposed method shows good potential to be further applied in the research on torsional eye movement and in relevant medical applications.