Microsaccades and Exploratory Saccades in a Naturalistic Environment

Abstract

Microsaccades, small saccadic eye movements made during fixation, might accompany shifts of visual attention, serve to refresh the retinal image, or have some other function. We tested the relative importance of these functions by recording exploratory saccades and microsaccades with a free head during a lane-change task in a simulated driving environment, accompanied by a simultaneous visual search task in which drivers searched for a target among similar distractors on a panel to the driver's right where an electronic display would normally be located. After training, observers performed a baseline run with the lane-change task only, followed by four dual-task runs and a final control run. In the dual-task condition, where more visual attention shifts occur, we found a significantly increased frequency of microsaccades along with an even larger increase in frequency of large exploratory saccades. However the proportion of microsaccades significantly decreased in the dual task, consistent with the idea of a common neurological origin for microsaccades and exploratory saccades.

1. Introduction

Visual perception takes place during fixations, with saccadic suppression interrupting it during saccades roughly three times per second (Zuber & Stark, 1966; Matin, 1974; Bridgeman, Hendry, & Stark, 1975; Irwin, 1992). During fixations, residual eye movements can be classified as drift, a slower curved movement following random-walk trajectories (Engbert & Kliegl, 2004), tremor (or nystagmus), a very low amplitude high-frequency oscillation layered upon drifts, and microsaccades (Rolfs, 2009; Martinez-Conde, Macknik, & Hubel, 2004; Martinez-Conde, Macknik, Troncoso, & Dyar, 2006; Martinez-Conde, Macknik, Troncoso, & Hubel, 2009). The latter show the same linear amplitude vs. peak velocity profile as ordinary saccades (the "main sequence," Zuber, Stark, & Cook, 1965) but with amplitudes too small to be measured without sufficiently sensitive oculometric equipment (criterion values in the literature range from 12 min arc to one degree) and occur involuntarily at an average rate of about one to two per second during idle fixation (Ditchburn & Ginsborg, 1953).

The role of fixational eye movements in visual perception has been debated for more than fifty years (Martinez-Conde, Macknik, & Hubel, 2004). For almost as long as we have known microsaccades exist (Ratliff & Riggs, 1950), we have also known that stabilized images, or images that counteract fixational eye movements to stimulate the same retinal receptors continually, fade from perception rather quickly (Ditchburn & Ginsborg, 1952; Pritchard, 1961). Fixational eye movements, however, include drifts and tremor in addition to microsaccades (Martinez-Conde, Macknik, & Hubel, 2004), which may contribute differentially to moving the retinal image to inhibit fading. Considering the difference in receptive field sizes associated with the peripheral and central retina, microsaccades may be necessary to refresh receptors in the periphery whereas drifts should be sufficient to do so in the fovea. Recent findings suggest that momentary decreases and increases in microsaccade frequency are correlated with the fading and reappearance, respectively, of a static grating in the periphery and parafoveal regions (Martinez-Conde et al., 2006). In particular, during viewing of low-contrast stimuli (peripheral and parafoveal), microsaccades tend to be more frequent before transitions to visibility and less frequent before periods of fade-out.

Also, microsaccades occur in central fixation tasks under conditions of lower than ordinary drift-induced retinal image slip (Engbert & Mergenthaler, 2006). Recent studies have found microsaccades to occur in the absence of percetpual fading. Poletti and Rucci (2010) found microsaccades being more corrective for preceding drifts during accurate fixation on a cue than during relaxed fixation on a region of the screen. Using a non-human primate sample Cui, Wilke, Logothetis , Leopold , and Liang (2009) found that microsaccade rate was correlated with the perceptual state of target visibility, and suggest that the measured microsaccade rate and direction are reliable indicators of the perception. A recent study on microsaccades in scene viewing (Mergenthaler & Engbert, 2010) showed microsaccade rates produced in a fixation paradigm to be correlated with microsaccade rates extracted from fixations in scene perception, indicating a common neurophysiological basis.

Although necessary to counteract peripheral retinal fading, microsaccades can affect visual acuity negatively in that they are associated with brief periods of saccadic suppression (Beeler, 1967), much like the loss of momentary visual information associated with ordinary, observable saccades (Zuber & Stark, 1966). Saccadic suppression is stronger in the fovea than in the periphery (Bridgeman & Fisher, 1990), which is probably why microsaccades tend to be inhibited during high acuity tasks (Winterson & Collewijn, 1976; Bridgeman & Palca, 1980), and why many recent studies using attentional cueing paradigms show decreases in the average microsaccade rate following the presentation of task-relevant visual information. In contrast, Ko, Poletti and Rucci (2010) found that microsaccades aid high acuity tasks, by precisely relocating the gaze of the observer. The authors argue that previous studies missed the critical window of time in which microsaccades can be shown to be helpful for high acuity performance.

Additionally though, the directions of microsaccades have been correlated with those of covert attention shifts (Hafed & Clark, 2002; Engbert & Kliegl, 2003; Laubrock, Engbert, & Kliegl, 2005; Turatto, Valsecchi, Tamè, & Betta, 2007) and may be an overt indicator of their presence. Engbert & Kliegl (2003) using an endogenous cueing paradigm (Posner, Nissen, & Ogden, 1978) measured microsaccade rates and direction, finding that microsaccades occurred more frequently and showed a significant directional bias toward the cue at a time in which central cues were most effective. These observations led Engbert & Kliegl to argue that microsaccades could objectively index covert orienting. Laubrock, Engbert, & Kliegl (2005) extended the microsaccade correlation with attention to exogenous cues. Rates of microsaccade occurrence were similar to those found in Engbert & Kliegl (2003), although the directional effects were somewhat stronger for exogenous orienting The bias away from the cue is suggestive of inhibition of return (Posner & Cohen, 1984), and other recent studies recording microsaccades also share this interpretation (Galfano, Betta, & Turatto, 2004; Betta, Galfano, &, Turatto, 2007). A physiological indication of another possible microsaccade function is microsaccade-contingent bursts of neuronal activity in primate visual cortex (Martinez-Conde, Macknik, & Hubel, 2002).

Most previous studies of the occurrence and function of microsaccades have concentrated on artificial laboratory tasks which, while well-defined, leave open the question of microsaccade function in more naturalistic environments. Here we combine recording of overall microsaccades and exploratory saccades with a simulated driving task (lane change task) and a surrogate reference task (Search task), to investigate the relationship between microsaccade rate and visual workload, under the assumption that vision is the single most important source of information for the driver (Lansdown, 2000). According to Wickens’ (2002) multiple resources model, human vision processes rely on a general pool of mental resources, from which all other processing codes, stages and modalities draw. By adopting such a model, we consider visual workload as a part of mental workload. According to the specifications of an ISO (International Organization for Standardization) standard under development, ISO 26022 (ISO/DIS 26022, 2007), the Lane Change Test is a simple and reliable method for quantitatively estimating secondary task demand in a driving context. The test consists of driving and changing lanes according to information provided on road signs. The SuRT (Search task) was chosen as a secondary task (Mattes, 2003). It requires participants to locate a target among visually similar distractors (visual demand) and then select the portion of screen containing the target (manual demand). The Search task requires visual perception and manual response: such activities, according to Wickens’ multiple resources model (Wickens, 2002), require the same mental resources as the driving task and will therefore be more likely to interfere, because of increased mental workoad. Thus, we believe the Search task is a good compromise between the need for a generic distractor task and the need for using ecologically valid distractor tasks rather than artificial tasks (e.g. arithmetic calculation, memory tasks). The issue of finding which are the best methods and measurement techniques for mental workload in a driving environment is still an open question. (Young & Regan, 2007).

Our research will show that microsaccades and exploratory saccades as well as fixations are not obviously affected by the use of IVIS (In-Vehicle Information System) while driving, because of the need to continuously shift gaze from the IVIS to the road and vice versa, but could provide useful information to understand drivers’ mental workload.

Athough theories of microsaccade function focusing on either low-level vision or attention shifts are not necessarily mutually exclusive, they make different predictions for how microsaccade rates would be affected by the Lane Change Test, depending on the nature of the secondary task. Based on the results from Martinez-Conde et al. (2006), Engbert & Mergenthaler (2006), Horowitz, Fine, Fenesik, Yugenson, & Wolfe (2007a) and Laubrock, Kliegl, Rolfs, Engbert (2010), we predict that more microsaccades should be found under single-task conditions than dual-task; the increasing presence of large saccades in the dual-task condition means fixations are briefer and retinal images are less likely to fade. On the other hand, attentional theories of microsaccade function (Engbert & Kliegl, 2003; Hafed & Clark, 2002) and theories of visual exploration and visual search (Otero-Millan et al., 2008) predict that microsaccade rates should increase as attentional load increases, because there are more shifts of attention in the dual-task condition.

Saccades and Microsaccades can’t be analyzed without considering fixations. Rate and duration are the two main parameters we considered for fixations. The fixation duration (time) or its approximate inverse, the fixation rate, is the most extensively used parameter in various studies, although its meaning is far from clear (De Waard, 1996). While fixation rate is a simple indicator of the amount of visual load, fixation duration is more complex since it depends upon task characteristics (Backs & Walrath, 1992). These authors found increased fixation time in self-terminating search compared to an exhaustive search. Wilson & Eggemeier (1991) stated that the duration of fixations was related to difficulty in obtaining and interpreting information from instruments. O'Donnell & Eggemeier (1986) report that an increase in workload is accompanied by increased fixation duration. Similarly Rayner & Morris (1990) found that fixation duration increases with the amount of information to be extracted from a target. On the other hand several authors found a decrease in fixation duration associated with a greater need for visual inspection because of greater scene complexity (Miura, 1986; Underwood & Radach, 1998). According to these authors, we should expect a decrease in fixation duration in the dual-task condition since more visual load occurs, and an increase in fixation rate as a function of visual load.

It is important to underline that when comparing different levels of visual workload, possible differences between the frequency and the proportion of microsaccades and saccades can be observed. Our research will keep in mind this distinction. Differences between frequency and proportion of microsaccades and exploratory saccades are expected. In the dual task, where more visual workload occurs, we predict an increased frequency of exploratory saccades together with an increased frequency of microsaccades due to the increased visual exploration and visual search generated by the secondary task (Otero-Millan et al., 2008), and shorter fixation durations (Miura, 1986; Underwood & Radach, 1998). Considering the proportion of microsaccades no predictions can be made because we could not find any similar study for a comparison.

This research has implications for understanding the roles of saccades and microsaccades in naturalistic environments, as well as applied research in automotive system design.

2. Method

The experiment of the present study has been previously described in Benedetto, Pedrotti, Minin, Baccino, Re, and Montanari (2011): a brief description follows.

2.1. Experimental plan

Fifteen participants performed a simulated Lane Change Test (LCT, ISO, 2007) in a fixed-base driving simulator. In the LCT, people are required to drive on a three-lane straight road and change lane when indicated by road signs. After a training session, each participant made six three-minute lasting trials of which the first (baseline) and last (control) one were simple single tasks (performing the LCT alone), and the other four were dual-task trials (Figure 1). Within the latters, a concurrent visual-manual secondary task was added to the LCT for simulating driver interaction with an IVIS: participants were required to locate and indicate targets on a touch screen mounted on the dashboard.

2.1. Eyetracking Apparatus

The aim of this study was to observe saccadic and microsaccadic behavior in a naturalistic environment. For that reason a monocular head-mounted eye-tracker was used (SMI X-HED) and participants’ heads were not stabilized using a chin rest, but were free to move. A five-point calibration was made for each participant; calibration was further checked with a laser-pointing device before each trial. The eye-tracker software recorded eye movements at a 200Hz frequency with tracking resolution < 0.1° and a gaze position accuracy < 0.5°. The resolution is the more important feature for our study, as we analyzed saccadic magnitudes rather than fixation positions. The eye-tracker was lightweight (80 g) and was mounted on a special baseball cap without visor (Figure 2-A). Room lighting was kept constant during all experiment trials.

2.2. Secondary Task

A two-column Search task was set up (Figure 2-B): participants were required to double-click as fast as possible with their finger on the portion (left or right) of the touch screen where a target (larger circle) is located. Twenty images with random localization of the target circle were created for each of the two difficulty levels: the easy ones had fewer distractors (small circles), while the difficult ones had more distractors. In both difficulty levels, targets and distractors had a diameter of 1.4 cm and 0.7cm respectively.

2.3. Dependent Variables

Theoretically saccades and microsaccades have different functions, but technically (for eye-tracking systems) they are very similar because they fall along the same main sequence (Zuber et al., 1965). While saccades alternate with fixations, microsaccades occur within fixation. They are involuntary (Ratliff & Riggs, 1950) small-amplitude eye movements (Ditchburn, 1973), which typically occur at a rate of one to two per second, and have amplitudes that are usually smaller than 1°. Technically, eye tracking systems, including the system we used for the experiment (SMI BeGaze 2) consider, collect and detect saccades and microsaccades in the same way since they both are characterized by stereotyped relationships between amplitude, duration and angular velocity. They also share the same triggering mechanism in the superior colliculus (Hafed, Goffart, & Krauzlis, 2009). Since they also overlap in amplitude, they cannot be differentiated completely by any physical algorithm. The event detection software used in the current study selects saccades as primary events using a velocity-based algorithm (e.g. Salvucci & Goldberg, 2000): saccades and fixations are computed and derived from the primary saccade events (SMI, 2009). As saccades and microsaccades are theoretically different (different functions for the visual system and different amplitudes) but technically similar, all saccadic events were grouped into two amplitude classes. Consistent with previous analyses (Hafed & Clark, 2002; Martinez-Conde et al., 2009), to differentiate microsaccades from exploratory saccades we chose a range of amplitude up to 1°. Choice of this amplitude limit for microsaccades is also informed by the microsaccade detection algorithm of Engbert & Kliegl (2003). In Fig. 3 of that paper, more than 95% of 9183 microsaccades are less than 1° in amplitude, though a few are larger. Thus with a 1° amplitude threshold we capture most microsaccades and omit very few. Microsaccades were defined in our data set as main-sequence events with amplitudes of 1° or less. The maximum amplitude and maximum duration correspond to a velocity of 100°/s. Informal surveys of our data using smaller microsaccade thresholds showed the same qualitative results as those given below, though the power of these surveys was lower due to a smaller number of events. Since the overall correlation of amplitude and duration was 0.66 in the baseline run, 0.78 in the dual-task runs, and 0.69 in the control run, the durations were not considered further. The upper limit for microsaccades represents the lower limit for exploratory saccades, which range from 1° up to 60°. The amplitudes of exploratory saccades were grouped into bins of 1°-10°, and each 10° increment up to 60°. Since microsaccades, saccades and fixation are highly dependent of one another it is necessary to collect them in conjunction. In face of that, fixation was defined as a minimum 80 ms period in which the gaze is still within a 100 px area (SMI, 2009). With the aim of allowing comparisons between this and other studies we included a detailed main sequence plot for all sacadic events (microsaccades and exploratory saccades) detected during the LCT (Figure 3).

In order to validate our hypothesis we collected and analyzed two main dependent variables: Saccade frequency and Microsaccade frequency. Saccade and Microsaccade frequency sum to the number of saccadic events that take place in each experimental run. From these we calculated Saccade and Microsaccade proportions, which are the ratios between each participant’s saccadic event frequency (grouped in amplitude ranges defined above) and their total. Two secondary dependent variables were collected: Fixation frequency, the number of fixation events that take place in each experimental run, and Average fixation duration, the period of time in which the gaze is fixed on a single location (for this analysis, each participant’s fixation duration values were averaged). All the variables were collected and analyzed for each of the 90 runs (15 participants x 6 runs), including all data points between the START and the END signs of the LCT (ISO/DIS 26022 , 2007): thus, each interaction and subsequent data gathering was referred to the same 3-minute periods of single- (baseline, control) or dual-task (1, 2, 3, 4) driving runs (Figure 1). Repeated measures ANOVA on the dependent variables was used with a Greenhouse-Geisser correction.

3. Results

3.1. Microsaccade and Saccade frequency

A repeated measures ANOVA carried out on microsaccade frequency (Figure 4) revealed a significant general effect (F (2,28)=6.67 p= .01) across the experimental runs. Since planned contrasts revealed no significant differences within the dual-task conditions [1- 2- 3- 4] and the dual-task SuRT conditions (easy and difficult), these values were averaged. A planned contrast on microsaccade frequency between baseline VS control was not significant, possibly indicating no changes due to practice or fatigue. Contrasts baseline VS dual-task (F (1,14)=6.92 p= .02) and control VS dual-task (F (1,14)=8.64 p= .01) revealed microsaccade frequency to significantly increase in the dual-task conditions. Though absolute frequency of microsaccades increased in the dual-task condition, the frequency of larger saccades increased even more, consistent with the theory that microsaccades serve to refresh the retinal image (Martinez-Conde et al., 2006; Engbert & Mergenthaler, 2006).

Exploring saccade distribution, a clustering analysis (K-means) was performed and two amplitude categories were found:

-

from 1° to 10° (labelled short);

-

from 10° to 60° (labelled medium-high).

A repeated measures ANOVA carried out on short saccade frequency revealed no significant effects. However, a repeated measures ANOVA carried out on medium-high saccade frequency (Figure 4) revealed a significant general effect (F (2,28)=69.63 p< .01) across the experimental runs. Since planned contrasts revealed no significant differences within the dual-task conditions [1- 2- 3- 4] and the dual-task SuRT conditions (easy and difficult), these values were averaged. A planned contrast on medium-high saccade frequency between baseline VS control was not significant, indicating no changes due to practice or fatigue. Contrasts baseline VS dual-task (F (1,14)= 76.61 p< .01) and control VS dual-task (F (1,14)=67.62 p< .01) revealed – as expected - medium-high saccade frequency to significantly increase in the dual-task conditions because of the nature of the dual-task (continuously shifting gaze from road to IVIS and vice-versa).

3.2. Microsaccade and Saccade proportion

Besides frequency, the proportions of microsaccades and saccades were calculated in order to assess the relative contributions of each saccade type in each condition. A repeated measures ANOVA carried out on microsaccades proportion revealed a significant general effect (F (2,28)=11.48 p< .01) across the experimental runs. Since planned contrasts revealed no significant differences within the dual-task conditions [1- 2- 3- 4] and the dual-task SuRT conditions (easy and difficult), these values were averaged. A planned contrast on microsaccades proportion between baseline and control was not significant. Contrasts between baseline and dual-task (F (1,14)=18.98 p< .01), and control VS dual-task (F (1,14)=12.83 p< .01) revealed microsaccades proportion to significantly decrease in the dual-task conditions, consistent with more demanding sensorimotor tasks in the literature (Winterson and Collewijn, 1976; Bridgeman and Palca, 1980). The saccadic distribution clustering adopted for saccades frequencies was maintained for studying saccade proportions. Considering the proportion, in the baseline condition microsaccades represent 49% of all saccadic events, short saccades 33%, medium-high saccades 17%, and saccades higher than 60° of amplitude 1%. In the dual-task condition microsaccades represent 34% of all saccadic events, short saccades 23%, medium-high saccades 40%, and saccades higher than 60° of amplitude 3%. In the control condition microsaccades represent 46% of all saccadic events, short saccades 34%, medium-high saccades 19%, saccades higher than 60° of amplitude1% (Figure 5). Repeated measures ANOVA was carried out within the two saccadic amplitude categories. Since planned contrasts revealed no significant differences within the dual-task conditions [1- 2- 3- 4] and the dual-task SuRT conditions (easy and difficult), these values were averaged. Results returned significant general effect within the short (F (2,28)=5.94 p< .01) and medium-high (F (2.28)=35.40 p<.01) categories. The planned contrast on short saccadic amplitudes baseline VS control was not significant. Contrasts baseline VS dual-task (F (1,14)=5.50 p< .05) and control VS dual-task (F (1.14)=9.61 p< .01) revealed short saccadic amplitudes to significantly decrease in the dual-task conditions. Planned contrast on medium-high saccadic amplitudes between baseline and control was not significant. Contrasts baseline VS dual-task (F (1,14)=45.69 p< .01) and control VS dual-task (F (1.14)=36.02 p< .01) revealed medium-high saccadic amplitudes to significantly increase in the dual-task conditions.

3.3. Fixation frequency and Average fixation duration

A repeated measures ANOVA carried out on Fixation frequency revealed a significant general effect (F (2,28)=45.35 p< .01) across the experimental runs. Since planned contrasts revealed no significant differences within the dual-task conditions [1- 2- 3- 4] and the dual-task SuRT conditions (easy and difficult), these values were averaged. The planned contrast on Fixation frequency between baseline and control revealed an increased frequency in the control condition, which is very close to significance (F (1,14)= 4.50 p= .052). Although more fixations were found in the control condition compared to the baseline, the result is not statistically valid. Contrasts baseline VS dual-task (F (1,14)=45.11 p< .01) and control VS dual-task (F (1,14)=50.40 p< .01) revealed Fixation frequency to significantly increase in the dual-task conditions. A repeated measures ANOVA carried out on Average fixation duration returned a significant general effect (F (2,28)=44.10 p< .01 across the experimental runs. Since planned contrasts revealed no significant differences within the dual-task conditions [1- 2- 3- 4] and the dual-task SuRT conditions (easy and difficult), these values were averaged. The planned contrast on Average fixation duration baseline VS control was significant (F (1,14)= 6.78 p< .05), indicating changes that could be attributable to practice. Since shorter fixation durations were found in the control condition. Contrasts baseline VS dual-task (F (1,14)=49.39 p< .01) and control VS dual-task (F (1,14)=48.85 p< .01) revealed Average fixation duration to significantly decrease in the dual-task conditions, consistent with Miura (1986) and Underwood & Radach (1998) who found a decrease in fixation duration associated with a greater need for visual inspection because of greater scene complexity.

In summary, analyzing frequency and proportion of microsaccades and saccades, different results were found. Whereas the frequency of microsaccades and exploratory saccades increases (medium-high saccades) in the dual-task condition, where more visual workload occurs, the proportion of microsaccades and short saccades rate decreases in the dual-task condition concurrently with an increase of exploratory saccades (medium-high saccades).

4. Discussion

An important differentiation here underlined is between the frequencies of saccadic and microsaccadic events and their relative proportions. Considering the frequency of microsaccades and exploratory saccades, our results are consistent with Otero-Millan et al. (2008). In the dual task, our results show an increase of microsaccade frequency concurrently with an even larger increase of exploratory saccades. On the contrary, considering the proportion of microsaccades and exploratory saccades, our results are consistent with Martinez-Conde et al., (2006) and Engbert & Mergenthaler, (2006), because smaller proportions of microsaccades occur in the dual task. However, we believe the distinction between frequency and proportion of events is important to keep in mind in interpreting other studies. Though microsaccades have been shown to correlate with attention shifts in certain very specific conditions (Hafed & Clark, 2002; Engbert & Kliegl, 2003; Laubrock, Engbert, & Kliegl, 2005; Turatto et al., 2007; Laubrock et al., 2010), their role remains in question for tasks requiring sustained periods of attention and a variety of skeletal motor responses. Some suggest, however, that microsaccades and attention are not associated (Kowler & Steinman, 1980; Tse, Sheinberg, & Logothetis, 2002; 2004; Horowitz et al., 2007a; 2007b, Laubrock et al., 2010).

There may be a generalized function of microsaccades, and in support of this idea single-cell studies have shown that visual cortical spiking in occipital regions is correlated with microsaccades in a context-specific manner. One lab showed increased spiking about 50 ms after a microsaccade while presenting oriented bars to the receptive field of otherwise passively fixating monkeys (Martinez-Conde et al., 2000; Martinez-Conde et al., 2002). In a complementary condition where perception is suppressed, during binocular rivalry, spikes are suppressed in a function peaking about 100 ms after a microsaccade (Leopold and Logothetis, 1998). In the dual-task condition of our study, the frequency of microsaccades may have increased even while large saccades were increasing, in order to gain a more frequent sampling during the post-saccade periods. Other evidence of increased intensity of attention includes larger average pupil sizes, and more frequent but briefer blinks (Benedetto et al., 2011). Thus many indicators of mental workload might work together to improve performance by keeping attention focused.

One of the most promising applications of eye tracking research could be in the field of automotive design. Sight is the most important source of information in driving (Lansdown, 2000), and several eye movement metrics can be used to provide a vehicle with the capacity to assess in real-time drivers’ visual behavior and prevent accidents. According to our results, study of microsaccades and saccades for system design in automotive research can be informative about the online monitoring of driver visual workload.

Although this study was carried out in a naturalistic environment, further applied research is needed to understand whether our dependent variables could be pulled out of laboratories and used in real driving condition where many other independent and latent variables occur.