Eye Movements and Attention in Visual Feature Search with Graded Targetdistractor-Similarity

Abstract

We conducted a visual feature search experiment in which we varied the target-distractorsimilarity in four steps, the number of items (4, 6, and 8), and the presence of the target. In addition to classical search parameters like error rate and reaction time (RT), we analyzed saccade amplitudes, fixation durations, and the portion of reinspections (recurred fixation on an item with at least one different item fixated in between) and refixations (recurred fixation on an item without a different item fixated in between) per trial. When targetdistractor-similarity was increased, more errors and longer RTs were observed, accompanied by shorter saccade amplitudes, longer fixation durations, and more reinspections/refixations. An increasing set size resulted in longer saccade amplitudes and shorter fixation durations. Finally, in target-absent trials we observed more reinspections than refixations, whereas in target-present trials refixations were more frequent than reinspections. The results on saccade amplitude and fixation duration support saliency-based search theories that assume an attentional focus variable in size according to task demands and a variable attentional dwell time. Reinspections and refixations seem to be rather a sign of incomplete perceptual processing of items than being due to memory failure.

Introduction

As common as visual search is in everyday life, as long is its history in scientific research. Whereas an abundance of literature exists relating measures of search efficiency like error rates and reaction times to attention, much less research has been undertaken to investigate search efficiency via eye movement parameters in regard to attention, and many questions are still open.

In the research presented here, we analyzed scan paths in visual feature search with graded target-distractorsimilarity in a typical search paradigm. We investigated visual search efficiency in terms of amplitudes of saccades, fixation durations, and number of reinspections/refixations in dependence of the experimental conditions target-distractor-similarity, set size, and target presence. Based on the results, we drew conclusions on how the three eye movement parameters relate to three hypothetically underlying attentional concepts: the attentional focus size, the attentional dwell time, and the depth of attentional processing. Finally, by proposing these relations, we were able to provide empirical support for the reaction time model of visual search STRAVIS (STRategies of VIsual Search; Müller-Plath & Pollmann, 2003), which is a modified version of Wolfe’s Guided Search (1994), and suggest improvements.

Search efficiency in terms of search rates, eye movement parameters, and attentional processes

In the tradition of visual search, search rates have been generally interpreted to draw conclusions regarding processes of visual attention (Treisman & Gelade, 1980). In general, flat reaction time curves with increasing set sizes argue for a more efficient search than steep reaction time curves (see Wolfe, 1996, for a review). Since in the reaction time slope the size of the attentional focus, the dwell time, and the total number of visits per item group are inextricably confounded, the search rate is of limited use for testing models of visual attention in detail, or for investigating interrelations between the focus size, the dwell time, the total number of visits, and conditions of the task. The analysis of eye movements in overt search might provide better insight here, provided that certain linking propositions between eye movement parameters and attentional processes hold. It is the central concern of the present study to test these propositions and to relate them back to (reaction time) models of visual search.

With respect to eye movements, efficient (overt) search shows up in (a) large saccade amplitudes, going along with (b) few saccades, and (c) short fixation durations (Jacobs, 1986). Furthermore, efficient search should mean (d) to fixate items/item positions at most once.

There exists a variety of models explaining search efficiency in terms of underlying attentional processes. Whereas Treisman’s Feature Integration Theory (FIT; Treisman & Gelade, 1980; Treisman & Sato, 1990) draws a clear distinction between attentional processes in feature and conjunction search, which is inconsistent with a variety of findings (e.g., Nakayama & Silverman, 1986), saliency-map-based models like Guided Search (GS; Cave & Wolfe, 1990; Wolfe, 1994) are able to account for a continuum of search efficiencies in feature search with variable target-distractor-similarity. However, neither their idea of a fixed attentional dwell time nor that of a focus containing only one item could be empirically confirmed. A modified version of GS that models search efficiency by an attentional focus of variable size and varying dwell times is STRAVIS, the main ideas of which have been outlined in Müller-Plath & Pollmann (2003) and Müller-Plath (2008; p. 318).(Compared to an older version (Müller-Plath & Pollmann, 2003), the process of attentional selection and the attentional focus has been conceptualized in more detail in the 2008 paper. The 2003 paper concentrates on reaction time modelling on a quite coarse scale) STRAVIS is a dynamical saliency map model with a strategic component that allows an explicit individual estimation of parameters from reaction times.

Although there might have been some conceptual confusion in the past (e.g., Itti & Koch, 2000), it should be clear that the “saliency” of a stimulus is per definitionem a psychological concept and not a physical one. Since the perception of physical attributes is modulated by attention (see e.g., Moran & Desimone, 1985; Kastner & Ungerleider, 2000), the saliency of a stimulus results from its physical contrast to its neighbours, as well as from the amount of selective attention paid to these contrasts. The more narrowly attention is focused (in space as well as in the amount of features attended to), the stronger modulation of perception is achieved. First, the model assumes that the observer can voluntarily adjust on how large an area he/she deploys how much attention, with the total capacity being limited. This area is termed “attentional focus.” Second, the attentional modulation of perception takes time. Thus, in order to achieve stronger perceptual modulation, attention has to be deployed longer. In visual feature search with homogeneous distractors, a target is detected if it is sufficiently salient. Therefore, if the target is physically dissimilar to the distractors, not much attention is necessary to achieve sufficient salience. Consequently, attention will be distributed widely (large focus; see Figure 1a. for a metaphorical illustration) and dwell only briefly. In the case of high target-distractor similarity, sufficient target saliency will be achieved only if attention is concentrated more narrowly in space (small focus; see Figure 1b,c) and dwells there for a longer time.

All models, including STRAVIS, assume that search is self-terminating in target-present trials and exhaustive in target-absent trials, i.e., sufficient processing depth of items inside the focus and a perfect memory for already visited item positions (but see Horowitz & Wolfe, 1998). Consequently, no refixations or reinspections should occur, which is in contrast to already existing eye movement studies on visual search (see e.g., Dickinson & Zelinsky, 2005, 2007; Gilchrist & Harvey, 2006; Hooge & Erkelens, 1996 for reinspections, and e.g., Findlay, Brown & Gilchrist, 2000; Hooge & Erkelens, 1998, 1999 for refixations).

In the present study, we manipulated search efficiency by varying target-distractor-similarity and set size. We measured error rates and reaction times in target-present and target-absent trials as well as the amplitudes of a representative saccade in every trial, the durations of a representative fixation, and the number of reinspections and refixations per trial. In order to draw conclusions about the three hypothetically underlying attentional concepts the attentional focus size, the attentional dwell time, and the depth of attentional processing we presumed the following linking propositions: Fixations are linked with allocating attention to the fixation point and the execution of a saccade is preceded by a covert shift of attention (Deubel & Schneider, 1996; Findlay & Gilchrist, 2003; Hoffman & Subramaniam, 1995). The saccade amplitude might then reflect how many items are checked in parallel (attentional focus size), and the fixation duration might indicate how much time is necessary to inspect one item or item group (attentional dwell time). The relation between reinspections/refixations and underlying cognitive processes is less clear. They might be linked with incomplete perceptual processing, leading to uncertainty in decision-making, or with incomplete memory (Peterson, Kramer, Wang, Irwin, & McCarley, 2001).

Amplitudes of saccades and the focus of attention

Following the above linking proposition, the longer the amplitude and the smaller the number of saccades, the larger should be the attentional focus. Here, we define the “size of the focus of attention” in terms of the number of items: Subjects try to adapt it to as many items as possible so that if one of them is the target it will still stick out (Müller-Plath & Pollmann, 2003). Synonyms in the literature are (beside others): “visual span” (O’Regan, Levy- Schoen, & Jacobs, 1983) or “zone of focal attention” (Motter & Belky, 1998a). Since the number of saccades depends on the focus size as well as on the number of reinspections/refixations (see the previous section), we regarded it as not a suitable measure of the focus size. We relied solely on saccade amplitudes.

Several eye movement studies found that saccade amplitudes decreased with increasing target-distractorsimilarity (e.g., Hooge & Erkelens, 1996, 1998, Näsänen, Ojanpää, & Kojo, 2001; Vlaskamp, Over, & Hooge, 2005). These findings are in line with predictions from models of visual attention for the focus size (or synonymous concepts). According to our notion of visual attention (see e.g., Müller-Plath & Pollmann, 2003; Müller-Plath, 2008; also Jacobs, 1986) it should be an efficient strategy in case of low target-distractor-similarity to process many items simultaneously (large focus of attention), because the target would be salient enough to stick out and the risk of missing a target would be low. In case of high target-distractor-similarity, it should be an efficient strategy to process only few items simultaneously in order to keep the risk of missing a target low (small focus of attention), because the target would be not salient enough to stick out from a larger group. The Attentional Engagement Theory (AET; Duncan & Humphreys, 1989), although theoretically different, predicts the same. As suggested by both lines of research (eye movement analyses and models of attention), we expected thus (i) decreasing amplitudes of saccades with increasing target-distractorsimilarity.

The set size also seems to have an influence on the amplitudes of saccades (e.g., Motter & Belky, 1998a, 1998b; Näsänen et al., 2001). Motter and Belky (1998a, 1998b) studied visual search and eye movements using rhesus monkey subjects. Their results showed an increase of the amplitude of saccades with increasing set sizes. Näsänen et al. (2001) replicated these results with human subjects and concluded that the visual span increased with increasing set sizes. In attention theories, the role of the set size in regard to the search efficiency is controversially discussed. On the one hand, Treisman and Gelade (1980) proposed no influence of set size on feature search. On the other hand, some models that include the computation of a saliency map suggest an (implicit) effect of the set size: Through a process of averaging perceptual contrasts across all items in the display, the perceptual salience of the target should be the higher the more homogeneous distractors are presented, even if the items do not physically change (Müller-Plath & Pollmann, 2003; Wolfe, 1994). Consequently, the attentional focus size might increase with increasing set size (see above). In the present experiment, we thus expected (ii) increasing amplitudes of saccades with an increasing set size.

Results for target-absent trials have rarely been interpreted in the eye movement literature. In visual search, the mean reaction time in target-present trials is usually shorter than in target-absent trials. A 1:2 ratio in search slopes is often interpreted as self-terminating or exhaustive search with focus size 1 (item) in both target-present and absent trials (e.g., Treisman & Gelade, 1980; Wolfe, 1994). However, this has never been confirmed directly. Since in our experiment the observer did not know anything about target presence when starting inspection, saccades before finding the target should not be influenced by its presence. However, when the target is within reach, the focus should be narrowed and the saccade shortened in order to fixate it (Zelinsky, Rao, Hayhoe, & Ballard, 1997). Although we only analyzed saccades that occurred early in the trial as representatives of the focus size (2^nd saccades, for details see the Methods section), a considerable number of them might be target saccades, especially in the easy condition with low target-distractor-similarity. We thus expected (iii) shorter amplitudes in target-present than in target-absent trials.

Fixation duration and the dwell time

Several studies report increasing fixation duration with increasing target-distractor-similarity (Hooge & Erkelens, 1996, 1998; Näsänen et al., 2001; Vlaskamp et al., 2005; see also the early meta-study by Moffit, 1980). According to the above linking proposition, the larger the duration of a fixation is, the larger is the proposed dwell time.

An open question is how the size of the attentional focus, i.e., the number of items to which attention is paid to simultaneously, is related to the dwell time. In terms of the associated eye movement parameters researchers have explored the relation in two opposite ways. On the one hand, in tasks with variable presentation time an increase of fixation duration was found to be associated with an increase of the number of inspected items (Mackworth, 1976; Salthouse & Ellis, 1980; Scialfa & Joffe, 1998). These results were interpreted as an increase in visual span or focus size, implying a positive correlation between attentional focus and dwell time.

On the other hand, using free viewing with unlimited presentation time, some studies showed an increase of fixation duration with increasing target-distractorsimilarity (e.g., Hooge & Erkelens, 1996, 1998), others found decreasing saccade amplitudes with increasing target-distractor-similarity (see above). Taken together, this argues for saccade amplitude and fixation duration to be negatively correlated. Theoretically, long fixations and small amplitudes might both reflect low target salience, implying a negative correlation between the attentional focus size and the dwell time (Müller-Plath & Pollmann, 2003). In the present experiment, which resembled the latter viewing conditions, we thus expected (i) increasing fixation durations with increasing target-distractorsimilarity.

An influence of set sizes on fixation durations is rarely reported. Motter and Belky (1998a, 1998b) found the fixation duration to be independent of the set size. Since some attentional models that include saliency maps suggest higher perceptual target salience with increasing set size when distractors are homogeneous, the dwell time might covary with the set size, provided one assumes that the dwell time depends on target salience. Linking fixation durations with dwell times, we expected (ii) decreasing fixation durations with increasing set size in the present experiment.

As mentioned above, results for target-absent trials are rarely reported in the visual search literature (Jacobs, 1986). Since in our experiment the observer did not know anything about target presence when starting inspection, fixations before finding the target should not be influenced by its presence. However, fixations on targets may take more time than fixations on distractors. Regarding the STRAVIS model, it seems plausible that part of the time c (see

Table 6. Model parameters in STRAVIS and STRAVIS 2.0.

for a brief description of STRAVIS’ parameters), reflecting the preparation and execution of the motor response, takes place during the target fixation. The fixation we chose as representative – the 2^nd fixation of each trial – will in a considerable number of trials be the target fixation in target-present trials. We thus expected (iii) longer fixation durations in target-present than in target-absent trials.

Reinspections, refixations, and the depth of attentional processing

In addition to eye movement parameters, analyzing scan paths gives information about the searching behaviour and efficiency in greater detail. Particularly reinspections and refixations can be interpreted in regard to attentional processes, e.g., incomplete perceptual processing of items, or in regard to the memory of the search path (e.g., Horowitz & Wolfe, 1998, 2001, and 2003; Kristjánsson, 2000; Peterson et al., 2001; McCarley, Wang, Kramer, Irwin, & Peterson, 2003; Dickinson & Zelinsky, 2005, 2007). A reinspection is defined here as a fixation of an item that has previously been visited with at least one different item visited in between. Synonyms in the literature are refixation, revisitation, or regressive saccade, whereas a refixation is defined here as the immediate revisitation of an item. Since the two types of recurred fixations might have different functions in decision-making on target absence or target presence, in contrast to other researchers (e.g., Dickinson & Zelinsky, 2005; Hooge & Erkelens, 1998; Peterson et al., 2001), we analyzed reinspections and refixations separately.

Reinspections are often reported in the literature but interpreted in different ways (e.g., Hooge & Erkelens, 1996; Dickinson & Zelinsky, 2005, 2007). A small number of reinspections can be associated with memorydriven models. However, many reinspections (Gilchrist & Harvey, 2006; Dickinson & Zelinsky, 2005, 2007) need not necessarily imply amnesic search. Peterson et al. (2001) suggested that participants might intentionally reinspect items because attention has prematurely left the item before it was adequately processed. They fitted three models of conjunction search which lead to different predictions about the distribution of reinspections based on the hazard function (the hazard function gives the conditional probability that an event will occur at a time t given that it has not occurred before t). Their first model assumed amnesic search, the second model proposed inadequate processing (miss), and the third model hypothesized conscious inadequate processing (miss + realization). Both, miss-model and miss + realization-model fitted the data better than the amnesic model.

The occurrence of refixations is reported often too, but researchers handle them differently. Some authors called them corrective saccades, attributed them to incomplete processing of objects, and excluded them from analysis (e.g., Dickinson & Zelinsky, 2005; Hooge & Erkelens, 1998, 1999). Peterson et al. (2001) summed up the durations of subsequent fixations on one item. Findlay, Brown and Gilchrist (2000) analyzed all second saccades in a conjunction search task and found that small corrective second saccades (refixations) occurred more frequently when the first saccade landed on a target rather than on a distractor. But taken together, refixations have rarely been analyzed in a systematic fashion.

As mentioned above, most models of visual search, including STRAVIS, postulate that each item (item position) is reviewed at most once and assume that visual search is self-terminating in target-present trials and exhaustive in target-absent trials (e.g., Duncan & Humphreys, 1989; Müller-Plath & Pollmann, 2003; Treisman & Gelade, 1980; Treisman & Sato, 1990; Wolfe, 1994; but see Chun & Wolfe, 1998, for an alternative account). Applying these models to overt search, reinspections and refixations should not occur.

In the present study we first wanted to provide evidence for the “Peterson conclusion” that visual search has a memory and reinspections are possible in memorydriven models (Peterson et al., 2001). Second, we wanted to analyze the function of reinspections and refixations in greater detail. Concerning the former, we chose experimental conditions in which a memory-driven model is highly plausible, i.e.: a) The display was systematically and statically organized (Gilchrist & Harvey, 2006); b) the items were presented until the subject responded (Horowitz & Wolfe, 2003); and c) set sizes were only four, six, or eight items (Dickinson & Zelinsky, 2007). If reinspections supported an amnesic model, we should observe them equally across all four target-distractorsimilarity levels. If they reflected decision uncertainty, their portion should increase with increasing similarity.

Assuming the latter, we expected (i) the highest portion of reinspections and refixations to occur in trials with the highest target-distractor-similarity. Supposed that increasing the set size from four to eight items would increase target salience and thereby facilitate decision making, we expected (ii) a larger portion of reinspections and refixations in smaller set size conditions. Concerning the difference between reinspections and refixations, we assumed that uncertainty about a negative decision arises mainly after having visited the entire set of items without finding a target, and that it would be most effectively reduced by going back to items with the highest probability of being a target - the peaks in a saliency map (Müller-Plath, 2008; Wolfe, 1994) - to ensure that there is no target that has been missed. Thus, we expected (iii) more reinspections in target-absent than in target-present trials. Assuming likewise that uncertainty about a positive decision arises mainly after having fixated the target too briefly, it would be most effectively reduced by an immediate refixation. We thus expected (iv) more refixations in target-present than in target-absent trials.

Method

Subjects. One male and four female undergraduate students (aged 24 to 29 years) participated in the experiment. All had normal or corrected to normal vision and no previous experience with eye movement experiments. They received course credit for their participation.

Apparatus. Stimuli were presented on a 20-inch monitor with a refresh rate of 85 Hz and a resolution of 1600 × 1200 pixels, controlled by a DELL Pentium IV PC running Matlab 6.5 using the Psychophysics toolbox (Brainard, 1997) and EyeLink toolbox (Cornelissen, Peters, & Palmer, 2002). Eye movement data were recorded with the SR Research Ltd. EyeLink II system operating at a sampling rate of 500 Hz and measuring participants’ gaze position with an error of less than 0.5 degrees of visual angle.

Subjects’ head position was fixed by a chinrest at a viewing distance of 55 cm. They answered with the right index or middle finger by pressing one of two external buttons. New trials were initiated by pressing the left mouse button with the left index finger. At the beginning of each block (i.e., every 24 trials) subjects performed a nine-point calibration and validation of gaze accuracy. Each single trial started with a fixation point in the middle of the screen for correcting postcalibration drift errors.

Stimuli. Distractor-items were red circular squarewave gratings with a spatial frequency of 3.2 cpd. Each item subtended 1.3° in diameter. The luminance was 10.0 cd/m² and the xy-CIE coordinates were (0.605, 0.369). The luminance of the grey background was 17.5 cd/m² with xy-CIE coordinates of (0.263, 0.299). Target-items differed from distractor-items in just one feature dimension: shape. Targets were ovals and differed in four similarity levels from the distractors. The appearance of target- and distractor-items is shown in Figure 2.

Four, six or eight items were adjacently located at twelve equispaced positions on an imaginary circle with a diameter of 8° visual angle (see Figure 3A). Hence, the display density was held constant in all set size conditions whereas its spatial extent varied. The starting position of the item row was random

Procedure. Each trial started with a central fixation cross. After 1000 milliseconds (ms) the search display appeared. Subjects were instructed to decide as quickly and accurately as possible whether a target was present or not and to indicate their decision with a button press. Apart from the instruction to fixate the central fixation cross until the search display appeared, no other instructions with regard to eye movements were given. Trials were blocked according to difficulty and set size condition. One block consisted of twelve target and twelve non-target trials in random order. In the course of the experiment subjects performed each block twice. In the first half of the experiment items were presented in increasing set size and similarity order starting with a block of similarity level one and four items and ending with similarity level four and eight items. The second half was arranged the opposite way. The experiment was divided into four sessions which took about one hour each. The total number of trials was 576 per subject (and 48 per condition).

Data analysis

We excluded all trials in which a subject either gave an incorrect answer or in which they did not initially fixate the display center.

Table B1. Absolute (and relative) frequencies of trials remaining after exclusion of error trials and trials without initial central fixation (summed across subjects). B2. Trials remaining for analysis of saccade amplitude and fixation duration (see text) (summed across subjects).

(in the Appendix B) shows the number of remaining trials.

For the latter criterion we defined a circular region around central fixation cross with a diameter of 0.8° visual angle. We analyzed eye movement data from display onset until 125 ms before button press: According to Geisler, Perry & Najemnik (2006), button presses occurring less than 125 ms after the onset of the last saccade were initiated before this last saccade. Thus, subjects should have completed their decision at this time.

In order to decide which item was fixated, we split up the display into twelve sectors of 30° each (see Figure 3B). The center of the sectors coincided with the item centers. Thereby fixations on the screen could be unequivocally assigned to items.

Analysis of saccade amplitude. We determined the amplitude of saccades by calculating the distance between two successive fixation locations.

In the present study, only a subset of saccades could sensibly be regarded as behavioral correlates of the focus size. Due to our display organization the following types of saccades did not reflect the focus size and were excluded: Saccades from the central fixation point to the item row (first saccades), reinspections, refixations, and saccades jumping from one end of the item row to the other.

We restricted our analysis to the amplitudes of second saccades in each trial. For trials in which the second saccade was a refixation, we used the third one. In general, later saccades were not a suitable measure because it differed remarkably between experimental factors how many saccades occurred at all in a trial (see Appendix A).

Figure 4 shows examples of included (A and B) as well as examples of excluded trials (C and D).

Analysis of fixation duration. The EyeLink II System allows online eye movement analysis of fixation durations. It typically shows a large variation between and within subjects. Consistent with our analysis of saccade amplitudes (see above), we restricted the analysis to fixations following the second saccade in each trial. We used all fixation durations distinguishable by the system and did not configure any thresholds.

Analysis of reinspections and refixations. We defined reinspections as those fixations that occurred again on a previously fixated item with at least one other item fixated in between. We defined refixations as fixations that immediately followed a fixation on the same item.

Results

Results are presented in two sections: First we give an overview on traditional visual search parameters like participants’ error rates and reaction times. Second, eye movement parameters like saccade amplitudes, fixation durations, and reinspections/refixations are analyzed.

Traditional search parameters

Error rates. For each subject, error rates were analyzed in the 4 × 3 × 2 = 24 experimental conditions. Analyses were run on these individual rates. Figure 5 points out the mean course of error rates depending on the factors similarity level, set size, and target presence.

Table 1. Three-way repeated measures analysis of variance for error rates.

gives the analysis of variance (ANOVA) results.

The main effects of similarity level and target presence were significant. As usual, error rates increased with increasing similarity. Interestingly, it increased also in the presence of a target. The two-way interaction similarity level × target presence was also significant: In higher similarity levels, there were more miss errors than false alarms. The large portion of misses indicates a tendency to abort the search prematurely in these conditions.

Reaction times. For each subject, reaction times (RTs) were analyzed in the 4 × 3 × 2 = 24 experimental conditions. Analyses were run on these individual means. Figure 6 points out the course of reaction times: As expected, they increased with the level of similarity, with the number of presented items, and in the absence of a target.

To ensure that the intended continuum of feature search efficiencies was replicated (Müller-Plath & Pollmann, 2003) we conducted a three-way repeated measures ANOVA (similarity level (SL) × set size (SS) × target presence (TP)). All main effects and all interactions were significant (

Table 2. Three-way repeated measures analysis of variance for reaction time.

) and the replication thus successful.

Eye movement parameters

Saccade amplitude. For all trials with at least two saccades, we determined the amplitude of the second saccade. Table B1 (in the Appendix B) shows the number of trials analyzed per condition. Analyses in the 4 × 3 × 2 = 24 experimental design were run on these individual means. Figure 7 shows the course of saccade amplitudes depending on the factors similarity level, set size, and target presence.

Table 3. Three-way repeated measures analysis of variance for saccade amplitude.

gives the ANOVA results.

The main effects of similarity level and set size were significant. As expected, saccade amplitudes decreased with increasing similarity, and increased with increasing set size. The proposed main effect of target presence marginally failed significance (p = 0.079). The two-way interaction similarity level × set size was also significant: The effect of set size on saccade amplitude seemed to be stronger in lower levels of similarity.

Fixation duration. We determined the average duration of second fixation. Analyses in the 4 × 3 × 2 = 24 experimental design were run on these individual means. Figure 8 points out the course of mean fixation durations depending on the factors similarity level, set size, and target presence.

Table 4. Three-way repeated measures analysis of variance for fixation duration.

gives the ANOVA results.

The main effects of similarity level and set size were significant. As expected, mean fixation duration increased with increasing similarity and decreased with increasing set size. Contrary to our expectation, main effect of target presence failed significance (p = .117).

Reinspections and Refixations. As mentioned above, we defined reinspections as events in which subjects returned to a previously fixated item with at least one fixation in between. Immediate refixations of one and the same item were analyzed separately. For each trial we calculated the portion of reinspections and refixations by dividing their number by the total number of fixations in that trial. For each subject, these portions were then averaged across trials in each experimental condition. Figure 9 points out the portion of reinspections and the portion of refixations depending on the three factors similarity level, set size, and target presence.

Table 5. Three-way repeated measures analysis of variance for portion reinspections and portion refixations.

gives the ANOVA results.

For both reinspections and refixations, the main effects of similarity level and target presence were highly significant. As expected, subjects returned to a previously fixated item or fixated the same item once again more often in trials where the target was more similar to the distractors. Further, as expected, target presence had different effects on reinspections and refixations: Whereas reinspections occurred more frequently in target-absent trials, refixations were observable more frequently in target-present trials. However - contrary to our expectationsset size did not show any reliable impact on reinspections or refixations. For reinspections, the two-way interaction similarity level × target presence was also significant.

Discussion

We realized a continuum of search efficiencies in visual feature search with graded target-distractor-similarity. In addition to effects on error rate and reaction time (RT), clear findings emerged from the analyses of saccade amplitude, fixation duration, and portion of reinspections and refixations. The factor similarity level consistently contributed to significant differences: As expected, we observed an increase in errors and RTs with increasing similarity. This was accompanied by a decrease of saccade amplitude, an increase of fixation duration, and an increasing portion of reinspections and refixations. The factor set size affected not only RTs but also the saccade amplitude and the fixation duration: As hypothesized, saccade amplitude increased and fixation duration decreased with increasing set size. Finally, the factor target presence produced differences in errors - more errors occurred in target-present trials (misses) - and RTs, which were longer in target-absent trials. As expected, there was a significant interaction indicating that the saccade amplitudes were longer in target-absent than present trials when target-distractor-similarity was low. The main effect marginally failed significance. Further, target presence had different effects on the portions of reinspections and refixations: As expected, more reinspections occurred in target-absent trials, whereas refixations occurred more frequently in target-present trials.

Saccade amplitude and the focus of attention

Saccade amplitude increased with decreasing target- distractor-similarity. This finding is in line with other studies (e.g., Motter & Belky, 1998a, 1998b; Näsänen et al., 2001) and consistent to the predictions of various models of visual search: For example, in a saliency-based model of visual search (e.g., Wolfe, 1994), the perceptual saliency of an item depends on its physical values compared to those of neighbouring items and on the perception of these physics being modulated by attention according to their task relevance (for a recent view on perceptual saliency, see Elazary & Itti, 2008). Thus, low targetdistractor-similarity results in high target saliency. Whereas GS (Wolfe, 1994) proposes items being checked oneby-one, and high target saliency showing up only in the probability of detecting the target in the first step being high, the modified saliency model STRAVIS (Müller-Plath & Pollmann, 2003; Müller-Plath, 2008) assumes high target saliency to result in spatially large search steps. From a theoretically different perspective, the same prediction is obtained: In a visual search model based on so-called “attentional weights” (AET; Duncan & Humphreys, 1989), items are given more resources the more they resemble the mental target template. The target item is most likely to enter visual short term memory (VSTM) in first instance when it has received high weight compared to the distractor items. Such a favourable weight distribution results when the target is not grouped with distractors and when the distractors are grouped together. Thus, irrespective of whether one conceptualizes the focus of attention as a spatial section of a saliency map or as a VSTM, it is predicted to be moved with large and few steps when target-distractor-similarity is low. In overt search, this is linked with large saccade amplitudes, which were observable in our results.

In line with our expectations and consistent to findings of other researchers (see e.g., Hooge & Erkelens, 1996; Vlaskamp et al., 2005), we found an increase of saccade amplitude with increasing set size. Two causes might be put forward for this. On the one hand, in a saliency- mapbased model of visual search target saliency is enhanced when more homogeneous distractors are added to the display. In AET, through a process called “weight linkage” (Duncan & Humphreys, 1989) the weight distribution becomes more favourable for the target when more homogeneous distractors are added to the display. Again, the prediction of both types of models is consistent with our finding. However, this result might also be due to the subject trading off accuracy for speed: The more items are presented, the more the subject might have been inclined to scan the display in large steps in order to maximize his/her chance to spot the target quickly. If there was no target detected at the end of the item row, the subject might have either wanted to go back to reinspect candidate targets, or respond “no” despite being uncertain. The first would show up in an increasing portion of reinspections with increasing set size, the second in an increasing rate of errors with increasing set size. Across all subjects, we observed neither. We conclude that the effect of set size on saccade amplitude was not due to a speedaccuracy trade-off.

Average saccade amplitudes were expected to be smaller in target-present trials than in target-absent trials because the saccade should be shortened when the target was inside the focus of attention compared to a regular search step. The effect of target presence was only marginally significant, probably due to saccades onto the target constituting only a portion of the saccades that were analyzed in the target-present condition. One might further ask whether the above discussed effects of similarity and set size might be an artifact of those reduced saccades. In order to exclude this, we re-analyzed the effect of similarity and set size on saccade amplitude with target-absent trials only. In the 4 × 3 = 12 design, the main effects of similarity level and set size remained significant (similarity level: F(3, 12) = 5.89, p = 0.010; set size: F(2, 8) = 17.5, p = 0.001). Thus, the above interpretation is maintained.

Fixation duration and the attentional dwell time

The fixation duration depending on the targetdistractor-similarity is in agreement with our expectations as well as with the literature. (Hooge & Erkelens, 1996; Vlaskamp et al., 2005): The more similar a target was to surrounding distractors, the longer the average second fixation in a trial. We had expected that the fixation duration would likewise be influenced by the set size: The larger the number of homogeneous distractors, the higher should be the physical impact on target salience, and the shorter should be fixations to decide upon its presence. This effect was confirmed, too. Since neither the error rate nor the number of reinspections increased with increasing set size, the effect of set size on fixation duration was obviously not due to a speed-accuracy trade-off (see above).

With regard to the question of how the saccade amplitude and the fixation duration are associated, we proposed a negative correlation. Observed correlations in the five subjects ranged between r = −.06 and r = −.45, only one of them reaching significance. Several researchers had proposed that during shorter fixations fewer items are checkable than during longer ones (Mackworth, 1976; Salthouse & Ellis, 1980; Scialfa & Joffe, 1998), implying a positive correlation between fixation duration and saccade amplitude. Although the two notions seem contradictory at first glance, they might be integrated in the framework of a dynamical saliency-based visual search model (for details, see the next section): If search difficulty is experimentally varied in a free viewing condition, saccade amplitude will decrease and fixation duration will increase with increasing difficulty, leading to a negative correlation. On the other hand, if the search difficulty is kept constant and the time of display presentation varied, the size of the attentional focus may increase with increasing fixation time, leading to a positive correlation.

The proposed main effect of target presence on fixation duration was, although descriptively present, not significant. As with saccade amplitudes, the portion of 2^nd fixations being target fixations might have been too small. Further, regarding the relation to attentional dwell time and the STRAVIS model, if attention dwells very long on the target in order to identify it, or if part of the constant time c is spent on the target, this time might as well be split up into two or more fixations on the target (refixations). This possibility, which would break up the strict linking proposition “dwell time = fixation time”, is discussed in the following paragraph.

Reinspections, refixations, and the depth of attentional processing

As expected, reinspections and refixations occurred more frequently with increasing target-distractorsimilarity. In similarity level four, about 30% of all fixations in a trial were reinspections or refixations. The increase of reinspections and refixations with increasing target-distractor-similarity argues against them being due to a failure of memory. This interpretation is in line with Peterson et al. (2001), who claimed them being a sign of incomplete perceptual processing.

Contrary to our expectation, the frequency of reinspections and refixations did not significantly decrease with increasing set size. Only for reinspections in similarity level four there might be a tendency in the data pointing in the expected direction (see Figure 8, left panel). Thus, even if more homogeneous items indeed enhanced target saliency, the effect did obviously not considerably reduce the need to fixate items more than once.

Finally, as expected, we found significantly more reinspections in target-absent trials and more refixations in target-present trials. Whereas reinspections seem to help ensuring that there was really no target, refixations might be used for validating that the just fixated item really was the target. The latter agrees with findings of Findlay, Brown & Gilchrist (2000), who showed that refixations occurred more frequently on target items. Taken together, reinspections and refixations seem to be necessary to reduce uncertainty arising from too superficial attentional processing at first visit. The significant interaction of similarity level and target presence on reinspections might indicate that at intermediate levels of similarity, “yes” decisions were easier to reach than “no” decisions (see Figure 9, left panel), whereas both were equally difficult in high similarity, and equally easy in low similarity.

Model of attention and eye movements in overt feature search: STRAVIS 2.0

Taken together, the findings might be accounted for by a quantitative model describing how visual attention operates in difficult feature search. The main ideas of the model STRAVIS have been mentioned above and outlined previously in Müller-Plath and Pollmann (2003) and Müller-Plath (2008; p. 318). The search was assumed to be self-terminating in target-present trials, which were exclusively modelled because of the lack of data on how the search is terminated when no target is present (Chun & Wolfe, 1996). According to the present results, the search seems to be neither necessarily self-terminating in target-present trials nor exhaustive in target-absent trials, in particular at high target-distractor-similarity. This suggests adding an “uncertainty” component, accounting for reinspections/refixations, and an “abortion” component, allowing for premature termination of item inspection. Both will be included in an extended model termed STRAVIS 2.0, which will be designed to comprehend target-present and absent trials.

In contrast to our original assumption, we found subjects going back to previously visited items, in particular when the physical contrast was low. STRAVIS should therefore be extended by a further strategic component. Not only does the subject adjust on how large an area he/she will deploy how much attention, but also is the attentional dwell time strategically traded off for the number of visits per item: When physical contrast in the display is low, it might be advantageous not to dwell too long on the items in a first run, hoping that the thereby achieved target salience will suffice, and if not, to go back and dwell again on candidate targets. In STRAVIS 2.0, these recurrent visits will be modelled by an additive “uncertainty” parameter u contributing to the number of dwells and movements of attention in a trial. Secondly, low target-distractor-similarity might cause the observer not to spend too much time on the search process but to abort the search with a guess after a considerable amount of time. This would show up in the observed high portion of miss errors in target-present trials. Although error trials are usually removed from modelling, there might be lucky guesses in target-present as well as absent trials after scanning only a fraction of the item row. In STRAVIS 2.0, the portion of omitted items will be modelled by a multiplicative “abortion bias” parameter b ≤ 1, affecting the number of dwells and movements of attention in a trial. Figure 10 illustrates the search course as proposed in STRAVIS 2.0 in contrast to STRAVIS. A deterministic version of the model equations is given below.

In a separate work, a stochastic version of STRAVIS 2.0 will be developed and tested how the number of parameters can be sensibly reduced in order to achieve a stable fit of the model to reaction times. Model fitting allows quantifying subprocesses of the search on the basis of reaction times without the drawbacks of the traditional slope analysis.

The general model equations read:

where E(RT_s,n,t) denotes the expectation of the reaction time across all trials at similarity level s (s = 1, 2, 3, 4), set size n (n = 4, 6, 8), and target presence t (t = 0, 1). (Note that even in a deterministic model, the target position and the starting position of the search are random). The movement time of attention from one item (group) to the next is represented by the parameter m, the (similarity dependent) dwell time by d_s and g(s, n, t) denotes the expected numbers of movements and dwells in dependence of similarity, set size, and target presence. In the deterministic version of STRAVIS and STRAVIS 2.0, the function g reads:

Here, 1_[ϕ] denotes the indicator function of a statement ϕ which is one if ϕ is true and zero otherwise. The half brackets indicate the ceiling function, rounding the argument to the next larger whole number. Table 6 gives an overview of the model parameters.

In relation to eye movements, STRAVIS 2.0 predicts that saccade amplitudes become smaller, fixation durations longer, and reinspections more frequent when targetdistractor-similarity is high. Although we found these patterns, the results on refixations suggests that the above proposed link between attentional dwell time and fixation duration should be extended: When the target is physically very similar to the distractors, a very long dwell of attention on each item would be necessary to reach a sufficient saliency of the target. Possibly, there is a point at which the proposed link between attentional dwell and fixation of the eye breaks down: It is well known that an optically stabilized image fades down quickly, and that this effect is much faster (usually less than 1 second) in the center than in the periphery (Gerrits, 1978). It might be counteracted by microsacccades (Martinez-Conde, Macknick, Toncoso, & Dyar, 2006). In our experiment, these would mostly be refixations. We thus suggest that the increased attentional dwell time with increasing similarity is reflected both by increased fixation times as well as by an increased frequency of refixations.

Conclusions

As expected, we found increased error rates and increased reaction times (RTs) when the target became more similar to the distractors. With regard to eye movements, increased target-distractor-similarity was reflected in decreased amplitude of saccade, increased fixation duration, and increased frequency of reinspections and refixations. Further, there was an effect of target presence on eye movements. In displays with a target, saccade amplitudes were shorter, and reinspections less frequent than in displays without.

The findings in target-present trials are compatible with our previously published model STRAVIS (Müller-Plath & Pollmann, 2003; Müller-Plath, 2008) on attentional processes in visual feature search when the model is extended by a further strategic component (STRAVIS 2.0): We suggest that the observer does not only strategically adjust the size of the attentional focus but also the attentional dwell time, with shorter dwell times being associated with more superficial perceptual processing and the necessity to visit items more than once. In addition, we propose including a bias parameter accounting for premature termination of the search.

The results provide further support to the following links between attention and eye movements in overt search: During a fixation, attention is centered at the fixation point, the execution of a saccade is accompanied by a shift of attention, and reinspections as well as refixations result from incomplete attentional processing. In addition, refixations might result when a very long attentional dwell can be realized only with the help of microsaccades (Martinez-Conde et al., 2006).

Finally, the findings demonstrate that the often observed RT increase with increasing target-distractorsimilarity and increasing set size is a result of three factors: a smaller focus of attention, longer attentional dwells, and more frequent revisits of items. Since in the RT slope these three factors are confounded, the RT slope cannot be interpreted as the processing time for each item (Treisman & Gelade, 1980; Wolfe, 2003). Fitting the model STRAVIS 2.0 to a set of RTs allows estimating these parameters separately. Visual search is obviously not in every case self-terminating or exhaustive. Moreover, it seems to have a good memory in static displays with regular item arrangement. Further research has to be engaged in order to generalize the present findings to conditions like unsystematic item organisation, more than one target, or heterogeneous distractors.