FIG 1 - uploaded by Sidney R Lehky
Content may be subject to copyright.
Schematic localization of visual pathways in the macaque monkey brain, indicating major visual areas along the dorsal pathway (blue arrows) and ventral pathway (red arrows). In this study, we recorded from lateral intraparietal cortex (LIP) and anterior inferotemporal cortex (AIT). Area LIP, located on the lateral bank within the intraparietal sulcus (IPS), is a high-level area in the dorsal pathway that shows visual responsiveness and is functionally important for spatial representations and eye movements. In the figure, the label for LIP is placed next to IPS, although LIP is actually located on the lateral bank of the sulcus and not visible from a surface view. Similarly, the label for MT is placed next to superior temporal sulcus (STS), although MT is actually located in the posterior fundus of the STS and not visible from a surface view. AIT is a high-level visual area in the ventral pathway, related to object recognition and memory. It extends from within the STS down past the anterior medial temporal sulcus (AMTS) to the ventral surface of the brain (not visible in this diagram), and anteriorly from the posterior medial temporal sulcus (PMTS). LuS, lunate sulcus; LaS, lateral sulcus; CS, central sulcus; AS, arcuate sulcus; PS, principal sulcus; MT, medial temporal area.
Source publication
Ventral and dorsal visual pathways perform fundamentally different functions. The former is involved in object recognition, whereas the latter carries out spatial localization of stimuli and visual guidance of motor actions. Despite the association of the dorsal pathway with spatial vision, recent studies have reported shape selectivity in the dors...
Contexts in source publication
Context 1
... central visual system includes several dozen visually responsive cortical areas, which may be placed in a hierarchy based on the pattern of their anatomical connections (Felleman and Van Essen 1991). These visual areas can be divided into two basic streams, a dorsal occipitoparietal pathway and a ventral occipitotemporal pathway ( Fig. 1), based originally on the patterns of behavioral deficits observed after brain lesions in both humans and monkeys (Farah 2004;Ungerleider and Mishkin 1982). The functional distinction between dorsal and ventral pathways can be defined in terms of a "what/where" dichotomy. The ventral, or "what," pathway is critical for object ...
Context 2
... analysis dendrograms based on the distance matrices in Table 1 are shown in Fig. 10. As expected from the larger response distances in AIT compared with LIP, the dendrogram for AIT is spread over a larger vertical scale than that of LIP. Interestingly, for AIT, the cluster analysis seems to have divided the eight shape stimuli into three groups, based on similarities in the populations' (relative) response patterns. ...
Context 3
... three-dimensional plot of the neural shape space produced by MDS is presented in Fig. 11A. The representations of the eight stimulus shapes within the AIT shape space are shown by the yellow, green, and purple dots, whose colors indicate the three groups of shapes previously picked out by cluster anal- ysis (Fig. 10A). The three groups remain clearly separated for the MDS analysis, as they were for the cluster analysis. For ...
Context 4
... three-dimensional plot of the neural shape space produced by MDS is presented in Fig. 11A. The representations of the eight stimulus shapes within the AIT shape space are shown by the yellow, green, and purple dots, whose colors indicate the three groups of shapes previously picked out by cluster anal- ysis (Fig. 10A). The three groups remain clearly separated for the MDS analysis, as they were for the cluster analysis. For LIP, the representations within its shape space are shown by blue dots. These are clumped near the origin because of the compressed scale of the LIP shape space relative to AIT (as would be expected from the relative magnitudes ...
Context 5
... two-dimensional plot of neural shape space is presented in Fig. 11B. The representations of the eight shapes within shape space for AIT are plotted as star-shaped points. As before, the yellow, green, and purple colors code the three groups of shapes previously found by cluster analysis. Also plotted is a Procrustes mapping (Borg and Groenen 1997) of the LIP configuration (circles) onto the AIT ...
Context 6
... two-dimensional correlation coefficient (Eq. 5) between the AIT and LIP configurations in Fig. 11B is r 2D 0.80. However, a permutation test on the goodness-of-fit value for the Procrustes mapping rejected the hypothesis that the AIT and LIP shape spaces were identical (P 0.25). This permu- tation test showed that the Procrustes fit between the AIT and LIP coordinates in shape space was not significantly better than could be ...
Context 7
... repeated exposure. Repetition effects have been widely reported in AIT (Brown and Bashir 2002;Fahy et al. 1993;Miller et al. 1991, Table 1. A: 3-dimensional plot of neural shape space configurations for AIT and LIP. For AIT, points representing 8 shapes are color-coded into 3 groups, yellow, green, and purple, derived from prior cluster analysis (Fig. 10A). Shape space configuration for LIP is indicated by blue points, clumped in region near origin because of com- pressed scale of LIP shape representations. B: 2-dimensional plot of neural shape space configuration for AIT, plotted to- gether with a Procrustes mapping of LIP configuration onto AIT configuration. Procrustes procedure ...
Context 8
... the AIT neural population, the cluster analysis divided the eight stimulus shapes into three groups (Fig. 10A). These groups had members who clearly resembled each other ("yel- low shapes": dominated by horizontal and vertical edges; "green shapes": variants of a hollow, doughnut-like ring; "purple shapes": triangular-like). In LIP (Fig. 10B), on the other hand, given the small distances separating population responses to different shapes ...
Context 9
... the AIT neural population, the cluster analysis divided the eight stimulus shapes into three groups (Fig. 10A). These groups had members who clearly resembled each other ("yel- low shapes": dominated by horizontal and vertical edges; "green shapes": variants of a hollow, doughnut-like ring; "purple shapes": triangular-like). In LIP (Fig. 10B), on the other hand, given the small distances separating population responses to different shapes (Table 1), the cluster analysis produced a compressed, poorly differentiated hierarchy, with less than 0.05 response distance separating seven of the eight ...
Context 10
... categorization is indeed built on top of the sort of grouping we observed in AIT, LIP would be expected to be poor at visual categorization, given the relatively undifferentiated re- sults of the cluster analysis for that area (Fig. 10B). That is, differences in responses to different shapes are so small that, in a noisy system, the shape space would need to be carved up into much coarser chunks to be reliably differentiated. There- fore given the small distances separating LIP population re- sponses to different shapes (Table 1), LIP is not only expected to do worse ...
Context 11
... MDS analysis reinforced the results of the cluster analysis. Projection of the MDS configuration to three dimen- sions (Fig. 11A) picked out the same three groups of shapes in AIT as did the cluster analysis. In the same figure, the LIP configuration appears bunched near the origin, again because of the small distances separating LIP responses to different ...
Context 12
... of the variance in the data for both LIP and AIT was accounted for by the three dimensions plotted in Fig. 11A (in fact, most variance can be accounted for by just 2 dimensions). While this is consistent with previous reports that the visual system is encoding shapes within a low-dimensional space ( Cutzu and Edelman 1996;Edelman and Duvdevani-Bar 1997;Op de Beek et al. 2001;Sugihara et al. 1998), we cannot place too much significance on this ...
Similar publications
Late ventral visual areas generally consist of cells having a significant degree of translation invariance. Such a "bag of features" representation is useful for the recognition of individual objects; however, it seems unable to explain our ability to parse a scene into multiple objects and to understand their spatial relationships. We review sever...
Visual perception is usually stable and accurate. However, when the two eyes are simultaneously presented with conflicting stimuli, perception falls into a sequence of spontaneous alternations, switching between one stimulus and the other every few seconds. Known as binocular rivalry, this visual illusion decouples subjective experience from physic...
Representing object position is one of the most critical functions of the visual system, but this task is not as simple as reading off an object's retinal coordinates. A rich body of literature has demonstrated that the position in which we perceive an object depends not only on retinotopy but also on factors such as attention, eye movements, objec...
To understand conscious vision, scientists must elucidate how the brain selects specific visual signals for awareness. When different monocular patterns are presented to the two eyes, they rival for conscious expression such that only one monocular image is perceived at a time. Controversy surrounds whether this binocular rivalry reflects neural co...
Viewed object-oriented actions elicit widespread fMRI activation in the dorsal and ventral visual pathways. This activation is typically stronger in the hemisphere contralateral to the visual field in which action is seen. However, since in previous studies participants kept fixation at the same screen position throughout the scan, it was impossibl...
Citations
... Notwithstanding these similarities, distinct differences exist between shape representations derived by the two pathways. Techniques with high temporal resolution, like event-related potentials, magnetoencephalography, and single-unit recordings, have shown that shape-selective signals in the dorsal pathway precede those in the ventral regions, suggesting that object representations in the dorsal pathway are not simply cascaded from computations carried out in ventral regions (Ayzenberg et al. 2023;Collins et al. 2019;Lehky and Sereno 2007;L. Liu et al. 2017;Regev et al. 2018). ...
The two visual pathways model posits that visual information is processed through two distinct cortical systems: The ventral pathway promotes visual recognition, while the dorsal pathway supports visuomotor control. Recent evidence suggests the dorsal pathway is also involved in shape processing and may contribute to object perception, but it remains unclear whether this sensitivity is independent of attentional mechanisms that were localized to overlapping cortical regions. To address this question, we conducted two fMRI experiments that utilized different parametric scrambling manipulations in which human participants viewed novel objects in different levels of scrambling and were instructed to attend to either the object or to another aspect of the image (e.g. color of the background). Univariate and multivariate analyses revealed that the large-scale organization of shape selectivity along the dorsal and ventral pathways was preserved regardless of the focus of attention. Attention did modulate shape sensitivity, but these effects were similar across the two pathways. These findings support the idea that shape processing is at least partially dissociable from attentional processes and relies on a distributed set of cortical regions across the visual pathways.
... The ventral pathway is concerned with object identity (Logothetis & Sheinberg, 1996) and the dorsal pathway with spatial cognition (Colby & Goldberg, 1999). However, some recent studies argued that representations associated with shape and location processing are present in both visual streams (Konen & Kastner, 2008;Lehky & Sereno, 2007;Sereno & Lehky, 2011;Sereno, Lehky, & Sereno, 2020). In a previous study using artificial neural networks (Han & Sereno, 2022a), we showed that the two cortical visual pathways for identity and space actively retained information about both identity and space independently and differently. ...
In our previous study (Han & Sereno, 2022a), we found that two artificial cortical visual pathways trained for either identity or space actively retain information about both identity and space independently and differently. We also found that this independently and differently retained information about identity and space in two separate pathways may be necessary to accurately and optimally recognize and localize objects. One limitation of our previous study was that there was only one object in each visual image, whereas in reality, there may be multiple objects in a scene. In this study, we find we are able to generalize our findings to object recognition and localization tasks where multiple objects are present in each visual image. We constrain the binding problem by training the identity network pathway to report the identities of objects in a given order according to the relative spatial relationships between the objects, given that most visual cortical areas including high-level ventral steam areas retain spatial information. Under these conditions, we find that the artificial neural networks with two pathways for identity and space have better performance in multiple-objects recognition and localization tasks (higher average testing accuracy, lower testing accuracy variance, less training time) than the artificial neural networks with a single pathway. We also find that the required number of training samples and the required training time increase quickly, and potentially exponentially, when the number of objects in each image increases, and we suggest that binding information from multiple objects simultaneously within any network (cortical area) induces conflict or competition and may be part of the reason why our brain has limited attentional and visual working memory capacities.
... Shape recognition is primarily facilitated by the ventral visual stream (Breitmeyer, 2014;Konkle and Caramazza, 2013;Lehky and Sereno, 2007;Pasupathy, 2006). Accordingly, the Decision condition engaged additional areas along the ventral visual stream to differentiate the circular targets from the ellipses. ...
Many goal-directed actions that require rapid visuomotor planning and perceptual decision-making are affected in older adults, causing difficulties in execution of many functional activities of daily living. Visuomotor planning and perceptual decision-making are mediated by the dorsal and ventral visual streams, respectively, but it is unclear how age-induced changes in sensory processing in these streams contribute to declines in goal-directed actions. Previously, we have shown that in healthy adults, task demands influence movement strategies during visuomotor decision-making, reflecting differential integration of sensory information between the two streams. Here, we asked the question if older adults would exhibit larger declines in interactions between the two streams during demanding motor tasks. Older adults (n=15) and young controls (n=26) performed reaching or interception movements towards virtual objects. In some blocks of trials, participants also had to select an appropriate movement goal based on the shape of the object. Our results showed that older adults corrected fewer initial decision errors during both reaching and interception movements. During the interception decision task, older adults made more decision- and execution-related errors than young adults, which were related to early initiation of their movements. Together, these results suggest that older adults have a reduced ability to integrate new perceptual information to guide online action, which may reflect impaired ventral-dorsal stream interactions.
... The primate ventral stream would be the ideal target for such investigation, but the anatomy and size of the monkey brain makes it difficult to compare more than a pair of visual cortical areas in a single study (e.g., see refs. 7,8,52 ). By contrast, the small size of the rodent brain, the spatial contiguity of rodent visual cortical areas, and the possibility of using a much larger number of animals than in monkey studies make it easier to probe and compare multiple visual regions (e.g., see refs. ...
Cortical representations of brief, static stimuli become more invariant to identity-preserving transformations along the ventral stream. Likewise, increased invariance along the visual hierarchy should imply greater temporal persistence of temporally structured dynamic stimuli, possibly complemented by temporal broadening of neuronal receptive fields. However, such stimuli could engage adaptive and predictive processes, whose impact on neural coding dynamics is unknown. By probing the rat analog of the ventral stream with movies, we uncovered a hierarchy of temporal scales, with deeper areas encoding visual information more persistently. Furthermore, the impact of intrinsic dynamics on the stability of stimulus representations grew gradually along the hierarchy. A database of recordings from mouse showed similar trends, additionally revealing dependencies on the behavioral state. Overall, these findings show that visual representations become progressively more stable along rodent visual processing hierarchies, with an important contribution provided by intrinsic processing.
... where x and s denote the mean and standard deviation of the observations (N in total). Based upon the use of kurtosis as a measure of neuronal selectivity (Lehky et al., 2005) and sparseness (Lehky and Sereno, 2007) in experimental neuroscience, we employ it as a measure of these properties in our model. An estimate of kurtosis obtained from responses of a single neuron to all stimuli is used as an estimate of image selectivity. ...
Predictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule and mimics the feedforward and feedback connectivity of the cortex. After training on image datasets, the models formed latent representations in higher areas that allowed reconstruction of the original images. We analyzed low- and high-level properties such as orientation selectivity, object selectivity and sparseness of neuronal populations in the model. As reported experimentally, image selectivity increased systematically across ascending areas in the model hierarchy. Depending on the strength of regularization factors, sparseness also increased from lower to higher areas. The results suggest a rationale as to why experimental results on sparseness across the cortical hierarchy have been inconsistent. Finally, representations for different object classes became more distinguishable from lower to higher areas. Thus, deep neural networks trained using a gated Hebbian formulation of predictive coding can reproduce several properties associated with neuronal responses along the visual cortical hierarchy.
... Moreover, related ideas have been put forth regarding non-visual domains such as auditory (Rauschecker and Scott, 2009), somatosensory (Dijkerman and de Haan, 2007), and language-related processes (Hickok and Poeppel, 2007;Saur et al., 2008). Despite their widespread influence, direct comparison of the dorsal and ventral pathways at the single-neuron resolution with matched stimuli is surprisingly scarce (Cheng et al., 1994;Lehky and Sereno, 2007), leaving open the exact nature of the differences and similarities between the two pathways. ...
... We directly compared single-unit selectivity between dorsal and ventral visual areas using matched stimuli, following the footsteps of a few pioneering studies (Cheng et al., 1994;Lehky and Sereno, 2007). Specifically, we reported the division of labor between extrastriate areas MT and V4 in stereopsis, as revealed by a novel manipulation of RDSs. ...
The division of labor between the dorsal and ventral visual pathways has been well studied, but not often with direct comparison at the single-neuron resolution with matched stimuli. Here we directly compared how single neurons in MT and V4, mid-tier areas of the two pathways, process binocular disparity, a powerful cue for 3D perception and actions. We found that MT neurons transmitted disparity signals more quickly and robustly, whereas V4 or its upstream neurons transformed the signals into sophisticated representations more prominently. Therefore, signaling speed and robustness were traded for transformation between the dorsal and ventral pathways. The key factor in this tradeoff was disparity-tuning shape: V4 neurons had more even-symmetric tuning than MT neurons. Moreover, the tuning symmetry predicted the degree of signal transformation across neurons similarly within each area, implying a general role of tuning symmetry in the stereoscopic processing by the two pathways.
... Shape recognition is primarily facilitated by the ventral visual stream (Breitmeyer, 2014;Konkle and Caramazza, 2013;Lehky and Sereno, 2007;Pasupathy, 2006). Accordingly, the Decision condition engaged additional areas along the ventral visual stream to differentiate the circular targets from the ellipses. ...
Many goal-directed actions that require rapid visuomotor planning and perceptual decision-making are affected in older adults, causing difficulties in execution of many functional activities of daily living. Visuomotor planning and perceptual decision-making are mediated by the dorsal and ventral visual streams, respectively, but it is unclear how age-induced changes in sensory processing in these streams contribute to declines in goal-directed actions. Previously, we have shown that in healthy adults task demands affect the integration of sensory information between the two streams and more motorically demanding tasks induce earlier decisions and more decision errors. Here, we asked the question if older adults would exhibit larger declines in interactions between the two streams during demanding motor tasks. Older adults (n=15) and young controls (n=26) performed a simple reaching task and a more demanding interception task towards virtual objects. In some blocks of trials, participants also had to select an appropriate movement based on the shape of the object. Our results showed that older adults made a similar number of initial decision errors during both the reaching and interception tasks but corrected fewer of those errors during movement. During the more demanding interception decision task, older adults made more decision- and execution-related errors than young adults, which were related to early initiation of their movements. Together, these results suggest that older adults have a reduced ability to integrate new perceptual information to guide online action, which may reflect impaired ventral-dorsal stream interactions.
Highlights
Older adults showed reduced performance in a visuomotor decision-making task
Initial decision errors were similar between young and older adults
Older adults were less likely to correct initial decision errors
More demanding movements were associated with earlier and less accurate decisions
... It is traditionally assumed that visual information is processed in a feedforward hierarchical model that simple visual features are coded at the primary (V1) or low-level visual cortex, and complex visual attributes converged at higher-order visual areas for perceptual output (Juan and Walsh, 2003;Ro et al., 2003;Briggs and Usrey, 2011;Klink et al., 2017). Specifically, different visual representations are formed along segregated parallel pathways, with object shape or form information processed by the ventral stream from area V1→V3→V4 and motion/spatial location signatures processed by the dorsal stream from V1→V2→V5 (Lehky and Sereno, 2007;Brown, 2009;Kravitz et al., 2011;Mercier et al., 2017). Similar ventral and dorsal visual streams are also defined in the cat after the homolog area 17, 18, 19, 21a and PMLS are equated with V1, V2, V3, V4, and V5 based on electrophysiology evidence and area-specific behavioral observations (Dreher et al., 1993(Dreher et al., , 1996bPayne, 1993;Wang et al., 2000Wang et al., , 2007Shen et al., 2006;Tong et al., 2011;Connolly et al., 2012). ...
Previous studies indicate that top-down influence plays a critical role in visual information processing and perceptual detection. However, the substrate that carries top-down influence remains poorly understood. Using a combined technique of retrograde neuronal tracing and immunofluorescent double labeling, we characterized the distribution and cell type of feedback neurons in cat’s high-level visual cortical areas that send direct connections to the primary visual cortex (V1: area 17). Our results showed: (1) the high-level visual cortex of area 21a at the ventral stream and PMLS area at the dorsal stream have a similar proportion of feedback neurons back projecting to the V1 area, (2) the distribution of feedback neurons in the higher-order visual area 21a and PMLS was significantly denser than in the intermediate visual cortex of area 19 and 18, (3) feedback neurons in all observed high-level visual cortex were found in layer II–III, IV, V, and VI, with a higher proportion in layer II–III, V, and VI than in layer IV, and (4) most feedback neurons were CaMKII-positive excitatory neurons, and few of them were identified as inhibitory GABAergic neurons. These results may argue against the segregation of ventral and dorsal streams during visual information processing, and support “reverse hierarchy theory” or interactive model proposing that recurrent connections between V1 and higher-order visual areas constitute the functional circuits that mediate visual perception. Also, the corticocortical feedback neurons from high-level visual cortical areas to the V1 area are mostly excitatory in nature.
... Efficient coding under criteria based on Shannon information theory has become an important concept organizing thinking about visual processing [2][3][4][5][6] . Neurophysiological studies have characterized sparseness and other measures of efficient coding across various areas of the visual cortex [7][8][9][10][11][12][13][14][15][16][17][18] , as well as the lateral geniculate nucleus 19 and retina 20 . In addition to vision, the concepts of efficient coding and sparseness have been applied to data from a variety of other domains, including audition 21 , olfaction 22 , somatosensation 23 , and memory 24 . ...
... The stimulus set had eight shapes that were simple geometric shapes. The data were originally published by Lehky and Sereno 12 . ...
When measuring sparseness in neural populations as an indicator of efficient coding, an implicit assumption is that each stimulus activates a different random set of neurons. In other words, population responses to different stimuli are, on average, uncorrelated. Here we examine neurophysiological data from four lobes of macaque monkey cortex, including V1, V2, MT, anterior inferotemporal cortex, lateral intraparietal cortex, the frontal eye fields, and perirhinal cortex, to determine how correlated population responses are. We call the mean correlation the pseudosparseness index, because high pseudosparseness can mimic statistical properties of sparseness without being authentically sparse. In every data set we find high levels of pseudosparseness ranging from 0.59-0.98, substantially greater than the value of 0.00 for authentic sparseness. This was true for synthetic and natural stimuli, as well as for single-electrode and multielectrode data. A model indicates that a key variable producing high pseudosparseness is the standard deviation of spontaneous activity across the population. Consistently high values of pseudosparseness in the data demand reconsideration of the sparse coding literature as well as consideration of the degree to which authentic sparseness provides a useful framework for understanding neural coding in the cortex. 2
... We further rule out that the curvature of the DV manifold and encoding of stimulus difficulty arise from sensory responses or spatial attention to stimuli. Parietal neurons respond to a variety of stimulus attributes in their RFs (Bisley et al. 2004;Janssen et al. 2008;Lehky and Sereno 2007;Sarma et al. 2016) and also show strong response modulations depending on the location of spatial attention (Bisley and Goldberg 2003). However, these response properties do not explain the reversal of firing rates. ...
Lateral intraparietal (LIP) neurons represent formation of perceptual decisions involving eye movements. In circuit models for these decisions, neural ensembles that encode actions compete to form decisions. Consequently, decision variables (DVs) are represented as partially potentiated action plans, where ensembles increase their average responses for stronger evidence supporting their preferred actions. As another consequence, DV representation and readout are implemented similarly for decisions with identical competing actions, irrespective of input and task context differences. Here, we challenge those core principles using a novel face-discrimination task, where LIP firing rates decrease with supporting evidence, contrary to conventional motion-discrimination tasks. These opposite response patterns arise from similar mechanisms in which decisions form along curved population-response manifolds misaligned with action representations. These manifolds rotate in state space based on task context, necessitating distinct readouts. We show similar manifolds in lateral and medial prefrontal cortices, suggesting a ubiquitous representational geometry across decision-making circuits.
