Human versus random search. The first four classification images (from left to right) show the classification images for observer L.K.C. across the four targets. To simulate a searcher who looks about randomly hoping to find the target by chance, the noise stimuli were sampled randomly. The result of averaging the noise pixels at these random fixation points is shown in the far right. The lack of significant structure in the classification image for the random searcher sharply contrasts with the obvious target-like structures generated by the observer. All the classification images are displayed using the same range of gray scales to highlight the relative pixel magnitudes across classification images. 

Context in source publication

Context 1

... mean equal to the mean (Beard & Ahumada, 1998). The statistically thresholded classification image corresponding to the classification image in Figure 3A is shown in Figure 3B. The statistically thresholded results for all observers and targets are shown in Figure 4. The first row in Figure 4 illustrates the four targets (circle, dipole, triangle, and bow tie) that observers were instructed to find in the stimulus. Each of the other rows shows the classification images for the three observers (L.K.C., U.R., and E.M.) for these targets. Each subject made an average of 3,000 fixations per target. Thus, each classification image is the result of averaging around 3,000 noise patches. To quantify the uncertainty associated with the shapes of these classification images, we bootstrapped (Efron, 1994) the averaging procedure and computed the boundaries of each of the resulting thresholded classification images as follows. To detect the boundaries of the statistically filtered classification images, the two largest regions in the statistically filtered image were detected using a connected- components algorithm. The outlines of each of these regions were then used to represent the boundary of the classification image. Bootstrapping was then used to verify the robustness of these boundaries. First, the ensemble of noise patches at the observer’s point of gaze was resam- pled (with replacement) 200 times. The image patches in each bootstrap sample (3,000 patches, on average) were then averaged together, statistically filtered, and processed to detect the boundaries. The boundaries of the resulting 200 classification images were then added together and superimposed on the classification images to reflect the stability of the boundary. The aggregate of all of the bootstrapped boundaries is shown superimposed in red in Figure 4. The most striking result of this analysis is the emergence of classification images (in Figure 4) that resemble spatial features of the target. Notice that the boundaries of most classification images from the bootstrap procedure are well defined, indicating that the shape information for these classification images is indeed reliable. (In fact, the full width of a bootstrapped contour at a given location gives an upper bound on the width of the 99.5% con- fidence interval about that location). This indicates that although gaze is being rapidly shifted about the image in an effort to find the target as quickly as possible, saccadic programming is being clearly influenced by spatial features in the noise with sufficient precision to generate these robust classification images. Also worthy of note is the fact that these classification images vary in a target-dependent manner within an observer (to varying degrees), and they also vary across observers for a given target. The data of L.K.C. and E.R., for example, show fairly dramatic changes across the targets, indicating that the gaze of these observers was attracted by features unique to a particular target. For example, this effect is most pronounced for observer L.K.C. who, for the case of the circle, seemed to be fixating at regions of high luminance but modifies the search template for the dipole search by fixating at regions that have a luminance profile that matches the horizontal edge of the dipole. This adaptation of the classification image to match some feature of the target is evident, albeit to a lesser extent, for observer E.M. who changes his strategy for the dipole, the triangle, and the bow tie. Observer U.R., in contrast, seemed to use a simpler heuristic, consisting of fixating bright regions of roughly the appropriate size. The interobserver variability is evident for most of the shapes. Even in the case of the triangle, in which the classification images look fairly similar, closer inspec- tion reveals some subtle but reliable differences. Observer L.K.C.’s gaze tends to land on the right side of bright regions that have a dark region further to the right, whereas observer U.R.’s gaze lands on the left side of bright regions that have dark regions further to the left. Both of these observers, however, are attracted to roughly circular bright areas on average, whereas observer E.M. clearly favors a more angular, elongated structure. We also simulated an observer who randomly looks about the stimulus hoping to find the target by chance (this behavior was largely consistent with our own introspection of behavior in this very difficult task; observers reported often being surprised when their gaze landed on the target). We did this by simply selecting random spatial coordinates from a uniform distribution with the constraint that the entire ROI surrounding the fixation point had to be within the image. The number of random fixations used was approximately equal to that comprising the samples from the human observers (around 3,000). The result of adding the noise pixels at the random fixation points is shown in the far right in Figure 5, where all the classification images in the figure are displayed using the same range of gray scales to highlight the relative pixel magnitudes across classification images. The lack of any image structure in the random sampling case and the ability to generate many classification images across subjects from the same set of 1/ f noise stimuli indicates that observers are not random searchers but are actually directing their fixations to regions that resemble some feature of the target. To verify that observers were indeed trying to follow our instructions to find the target quickly, we analyzed the histograms of the fixation duration (time per fixation) and saccadic magnitudes for observers’ eye movements for this experiment. A relatively short fixation dwell time (mean = 0.25 s) and a wide range of saccadic magnitudes (mean = 3 deg, SD = 1.8 deg) confirmed that our observers were indeed trying to survey the range of the stimulus area quickly and were not fixating on particular regions for an inordinately long time. These statistics reflect previously reported measures of fixation durations and saccade magnitudes in natural viewing tasks (Duchowski, 2002; Rayner, 1998; Yarbus, 1967). Traditionally, experiments using the classification image analysis have used additive white Gaussian noise as the masking stimulus. White noise images, being spatially uncorrelated by design, do not contribute to any artificial structure to the classification images. In our experiments, we used 1/ f noise as the stimulus because the presence of many large-scale, target-like salient features inherent in the noise structure made it an effective masker. Due to the correlated nature of 1/ f noise, the resulting classification images are not unbiased linear templates. To obtain the true unbiased linear estimator, we can apply a prewhitening filter (Abbey & Eckstein, 2000) to the classification images obtained using 1/ f noise. An example of the dipole classification image for observer L.K.C. before and after prewhitening is shown in Figure 6. However, even the classification images obtained using the prewhitening filter does not reflect the true unbiased template in our experiments. This is because, unlike the experimental situation in psychophysics, the contribution of each pixel to the creation of the classification image is shift variant across trials (fixation points in this case) due to several factors. First, oculomotor precision and measurement errors inherent in eye movements and their recording result in spatial uncertainty in the exact location of an observer’s fixation. Second, even if we ignore errors in recording fixations and simply assume that an observer was using only a single visual feature to succeed in the search task, there is no guarantee that the observer will precisely fixate the same location on this feature every time. For example, assume that the observer always looked for black triangles in the case of the B bow-tie [ search. During search, the observer could decide to fixate the left triangle in some trials and the right triangle on others. Thus, the noise samples extracted around these fixations are not necessarily perfectly aligned across trials, resulting in spatially blurred classification images. However, this does not imply that observers are unable to use precise shape information. In a related study (Beutter, Eckstein, & Stone, 2004), subjects performed an 8-AFC contrast discrimination task and classification images were generated using a saccade-contingent analysis and an 8-AFC perceptual decision framework. The resulting classification images for these two cases were not found to be significantly different from each other, indicating that saccade mechanisms can indeed use precise shape information to guide search. Analysis of stimuli at the observer’s point of gaze can provide an understanding of strategies used by observers in visual tasks. Although existing proof for the guided saccade-targeting hypothesis is supported using efficiency of task performance or reaction times, in this paper we demonstrated that classification image analysis and accu- rate eye tracking can be used in conjunction to reveal shape cues that guided saccades in a difficult visual search task. Our results indicate that even in very noisy stimuli, human observers are not random in their search strategy; instead, they make directed eye movements to regions in the stimuli that resemble some structural feature of the target. Given the difficulty of the task and the rapidity at which fixations are made, we find it remarkable that the visual system seems to analyze spatial structure in the low-resolution periphery and direct fixations to regions that resembled some spatial feature of the target. The goal of this study was to investigate the influence of structural cues in visual search. However, reverse correlation analysis of noise stimuli at the point of gaze of observers has also been used to provide valuable ...