A Simple(r) Tool for Examining Fixations

Abstract

This short paper describes an update of A Simple Tool For Examining Fixations (ASTEF) developed for facilitating the examination of eye-tracking data and for computing a spatial statistics algorithm that has been validated as a measure of mental workload (namely, the Nearest Neighbor Index: NNI). The code is based on Matlab® 2013a and is currently distributed on the web as an open-source project. This implementation of ASTEF got rid of many functionalities included in the previous version that are not needed anymore considering the large availability of commercial and open-source software solutions for eye-tracking. That makes it very easy to compute the NNI on eye-tracking data without the hassle of learning complicated tools. The software also features an export function for creating the time series of the NNI values computed on each minute of the recording. This feature is crucial given that the spatial distribution of fixations must be used to test hypotheses about the time course of mental load.

Introduction

Eye-tracking has been used in Human Factors/Ergonomics (HF/E) studies for gathering information on human cognition since the seminal work by Fitts (Fitts, Jones and Milton, 1950). Among the great deal of indicators provided by eye-tracking systems, the geometry of the scanpath has been considered one of the most valuable for studying human interaction with complex systems both qualitatively and quantitatively. For example, randomness in visual scanning has been considered informative about the mental workload (Ephrath et al., 1980; Harris et al., 1986) and for many years such a measure of “entropy” has been proposed as an elective measure of that construct. Unfortunately, the “entropy” approach is totally dependent on a priori definition of Areas Of Interest (AOI), thus excluding eye movements outside those regions and limiting the application in real-world settings. Another in-depth analysis of scanpath has been suggested by Groner et al. (1984) and Menz and Groner (1985) by distinguishing between local and global scanpaths, where local scanpaths are those in immediate spatio-temporal neighborhood and global scanpaths refer to the distribution of fixations on a macro-time scale. More recently, Di Nocera, Camilli and Terenzi (2007) proposed a derived measure of mental workload based on the application of spatial statistics called Nearest Neighbor Index (NNI: Clark and Evans, 1954) to the whole distribution of fixations within a time-frame. Typically, the time-frame is one minute (see Di Nocera, Ranvaud and Pasquali, 2015), because for getting unbiased values more than 30 points are usually considered the threshold for computing the NNI. However, in our experience with eye movement data, about 50 fixations is a more realistic sample size. Furthermore, for the purpose of estimating changes in mental load, the one-minute window is quite suitable in most settings. The index is defined as the ratio of the average of the observed minimum distances between fixation points and the mean distance that one would expect if the distribution of fixations were random.

The NNI index has been repeatedly found varying significantly with the taskload imposed, pointing to its utility as an index of mental workload. The direction of the effect is related to the type of cognitive load imposed to the individual. Camilli, Terenzi and Di Nocera (2008) showed that the temporal demand leads to higher NNI values due to a more dispersed pattern of fixations(promptness to incoming stimuli is therefore maximized),whereas visuo-spatial demand leads to lower NNI values due to the fixations clustering (mental operations involved in the spatial task prevent the use of visual and spatial resources needed by ocular exploration).

Since its introduction, the NNI has gained popularity in the field, and convergent evidence for its validity has been therefore provided (e.g. Dillar et al., 2014; Moacdieh and Sarter, 2015). The NNI has been also featured in relevant textbooks like Holmqvist et al.’s (2011) and Wickens et al.’s (2013), thus making very likely its future use in the HF/E community. Finally, in ecological settings the index is also a valid alternative to ocular indicators like the pupil diameter that, based on Beatty’s (1982) work, is commonly considered as reflecting the cognitive load dynamics, but it is nevertheless affected by changes in luminance.

In order to facilitating other researchers when computing the NNI, a side project called ASTEF (acronym of “A Simple Tool For Examining Fixations”) was devoted to providing free and easy-to-use software tools.

The original ASTEF package (Camilli et al., 2008) was coded in C# and has been since then distributed for free to the research community. The software was developed both for facilitating the examination of eye-tracking data and for computing the NNI. The package was coded for running only on MS Windows machines and provided many tools for manipulating eye-tracking data. That version is no longer maintained and, in the tool that it is presented here, we did not include many functionalities that can be found in both commercial and open-source software solutions for eye-tracking (e.g. Dalmaijer, Mathôt and Van der Stigchel, 2014; Krassanakis, Filippakopoulou and Nakos, 2014). Indeed, ASTEF is specifically aimed at analyzing the spatial distribution of fixation as we have indicated in our publications. Of course there are plenty of software suites that calculate comprehensive sets of spatial statistics. Some of them are dedicated to particular domains (e.g. CrimeStat; Levine, 2013), whereas others are routines developed for data analysis programming languages such as R. However, R requires some programming skills to be used and software from other domains are not suited for the specific needs of the researcher working with eye-tracking data.

ASTEF was developed for making it easy for any researcher and practitioner to compute the NNI on eye-tracking data without the hassle of learning complicated tools. At the same time, we wanted to share code that could be accessible to as many researchers as possible. For that reason, we decided to use MathWorks’ Matlab, which is widespread in research laboratories, has a smooth learning curve, and it is suitable also for people with little programming background. Strengths of this multi-platform programming environment are the ease of manipulation of matrices and the generation of plots (both features required for computing spatial statistics), not to mention the availability of many “toolboxes” designed for psychologists, such as the Psychophysics Toolbox (Brainard, 1997).

Code

ASTEF is currently distributed on the web (http://www.astef.info) as an open-source project in order to allow other researchers to use and improve the code. The repository hosting service is GitHub, the largest and keenest open-source community in the world. The code is based on Matlab® 2013a. Source code is released under the new BSD license, a permissive free software license allowing commercial use, modification, and redistribution as long as the original authors of the code are cited in derivative works.

The source code is composed of independent modules for facilitating the integration of new features and the reuse of the code. Particularly, the NNI computation module is separated from the information presentation application; therefore, the NNI module could be easily imported into other projects. As an example, the NNI computation module was used for computing the index in real-time during an experimental session with the aim of implementing adaptive automation strategies (Proietti Colonna et al., 2015). This new implementation of ASTEF also features an export function for creating the time series of the NNI values computed on each minute of the recording. Indeed, as reported by Di Nocera, Ranvaud and Pasquali (2015), “the classification of point patterns as clustered, regular or random is only made for convenience: it is a snapshot in time and there is always a continuum among these categories, because a spatial pattern is the result of a process continuously evolving over time” (p. 468). Therefore, the NNI must be used to test hypotheses about the time course of a phenomenon.

Interface design

The new Graphical User Interface (GUI) has been designed according with the most straightforward workflow for data-analysis and optimized through basic dialogue principles (ISO 9241-110). Particularly, we took into account the suitability for the task, the selfdescriptiveness, and the conformity with user expectations. During this process, several functions (previously available in the first implementation of ASTEF) were eliminated for avoiding any unnecessary complexity. Only those features consistent with the idea of a very simple tool for examining the scanpath and computing the resultant spatial indicator were retained.

All the design choices were aimed at minimizing potential inconsistencies in the underlying structure, thus reducing the cognitive complexity of a system, because the interface functioning is matched to the user’s’ goals.

Appearance of the new GUI (see figure 1) was developed according with the minimalist Flat 2.0 design, which is defined by the absence of glossy, skeuomorphic and/or three-dimensional visual effects to the graphic elements (Turner, 2014). Cues such as borders, color, size and consistency were added for suggesting the clickability and functionality of the interactive components. Although this design style has been recently criticized (see Burmistrov et al., 2015), the clean look of the interface suits quite well the simplicity of this new implementation.

As reported above, the GUI was developed for supporting and facilitating the workflow: from left to right the user can load the background picture and the fixation data, visualize the scanpath minute by minute, monitor the NNI variation, and save the output of the analyses (i.e. heatmap, timeseries, graph). Generally speaking, the left pane is for the input and the right pane is for the output.

As a first step, the user must load an input file containing fixation data by clicking on the blue button “LOAD DATA”, which is located in the left panel. The input file has the following format convention (simple text, space-delimited):

• 1024 768
• 1083 369 482
• 1684 388 546
• 1856 359 589
• 2264 337 684
• …

where first row indicates the display geometry and resolution (in pixels) of the recording session and the successive rows represent the fixation data and include a timestamp (in milliseconds), X and Y coordinates of each fixation (in pixels). The display resolution is a very important piece of information, because all computing addresses data located in that very space.

As soon as the file has been loaded, the scanpath of the first minute of recording is plotted in the left panel and the NNI value is computed and plotted in the right panel. Optionally, the user can load a background image for the fixation frame by clicking on the appropriate button. The image will be automatically resized to fit the frame, but no consistency check will be performed between fixation data and background image.

Once fixation data are loaded, the user can browse the scanpath minute by minute by using the two arrows located below the fixation frame. The current minute under inspection is shown between the arrows. The scanpath is updated along with the NNI graph by highlighting the current value in the curve. The circle representing the current NNI value will be green-colored if the value has been computed with enough fixations (as a cautionary limit, it has been set to 50 points). Instead, a red-colored circle will indicate that the NNI value may be unreliable.

Three buttons for managing outputs are located underneath the NNI plot:

• the first button allows the user to save a text file with the time series of the NNI value computed each minute using the convex hull and the Donnelly adjustment (Donnelly, 1978);
• the second button saves a Portable Network Graphics (PNG) file of the NNI plot shown in the right pane;
• the third button allows saving (after visualization) a heatmap drawn from all the fixations of the input file.

Since the heatmap computation may take long time to draw, a progress bar is shown. Locations with higher density of fixations will be red-colored; transitions to yellow and blue will indicate a decrement in density. After visualization the heatmap image can be either saved as PNG or discarded.

Conclusions

Researchers interested in using the distribution of eye fixations as an indicator of mental load dynamics have three main objectives: loading their datasets, computing the time series of the dispersion index (namely, the NNI), and plotting/exporting the time series for further analyses or documentation. The code we have presented here has no bells and whistles, but matches exactly the workflow for computing and plotting the NNI values, while visualizing the scanpath. Design choices were based on our own laboratory experience and on the requests made by colleagues. Of course, the open-source model of distribution we have adopted should encourage other researchers to adding features that could be needed (e.g. K-nearest-neighbor classification algorithms). Other desired features may include the implementation of fixation detection algorithms for loading gaze data instead of fixations. However, we believe that many functionalities for manipulating eye-tracking data can be already found in commercial and open-source software (which is abundant) and we therefore favored sharing with the community A Simple(r) Tool For Examining Fixations.