ArticlePDF Available

Spatial Transformations in the Parietal Cortex Using Basis Functions

Authors:
Alexandre Pouget
Alexandre Pouget
Alexandre Pouget
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Terrence J. Sejnowski
Terrence J. Sejnowski
Terrence J. Sejnowski
  • This person is not on ResearchGate, or hasn't claimed this research yet.
Download full-text PDF
Read full-text
Download citation

Abstract and Figures

Sensorimotor transformations are nonlinear mappings of sensory inputs to motor responses. We explore here the possibility that the responses of single neurons in the parietal cortex serve as basis functions for these transformations. Basis function decomposition is a general method for approximating nonlinear functions that is computationally efficient and well suited for adaptive modification. In particular, the responses of single parietal neurons can be approximated by the product of a Gaussian function of retinal location and a sigmoid function of eye position, called a gain field. A large set of such functions forms a basis set that can be used to perform an arbitrary motor response through a direct projection. We compare this hypothesis with other approaches that are commonly used to model population codes, such as computational maps and vectorial representations. Neither of these alternatives can fully account for the responses of parietal neurons, and they are computationally less efficient for nonlinear transformations. Basis functions also have the advantage of not depending on any coordinate system or reference frame. As a consequence, the position of an object can be represented in multiple reference frames simultaneously, a property consistent with the behavior of hemineglect patients with lesions in the parietal cortex.
Content may be subject to copyright.
Content uploaded by Terrence Sejnowski
Author content
All content in this area was uploaded by Terrence Sejnowski on Jan 01, 2015
Content may be subject to copyright.
Head-centered
coordinates
Eye
position
Head
position
Shoulder
position
Arm-centered
coordinates
Body-centered
coordinates
Joint
coordinates
Eye-centered
coordinates
Movement
Visual Target
Arm
position
B
A
Activity
Retinal Position (Deg)
e = 0
e = 20
x
x
-10
0
20
40
60
-40 -20 0 20 40
e
y
e
x
0
0
Activity
r (Deg)
x
0 40-20-40 20
Activity
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
r
x
e
x
e = -20
e = 0
e = 20
x
x
x
e
x
r
x
e
x
r
x
Eye Position
Units
Hidden Layer
e
x
r
x
e
x
r
x
r
x
a
x
Retinotopic
Map
Head-Centered
Map
Head-Centered Retinotopic
e
x
r
x
e
x
r
x
e
x
r
x
e
x
r
x
e
x
r
x
e
x
r
x
-10
0
20
40
60
-40 -20
rx
0 20 40
0
10
20
30
-40 -20 0 20 40
ry
-5
0
10
20
30
-40 -20 0 20 40
ry
Response (spikes/s)
Retinotopic Position (°)
-10
0
20
40
60
-40 -20 0 20 40
ry
0,0
0,20
0,-20
20,-20
-20,-20
20,0
-20,20
A B
C D
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
x
y
e
e
x
y
e
e
0
5
10
15
20
25
Number of cells
θ (Deg)
-80< -72 -60 -48 -36 -24 -12 0 12 24 36 48 60 72 >80
θ
e
x
e
y
Firing Rate
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
R
x
y
R
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
-40
-20
0
20
40
r
x
e
x
r
x
e
x
r
x
e
x
r
x
e
x
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
a
y
a
x
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
a
y
a
x
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
a
y
a
x
a
y
a
x
a
y
a
x
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
a
y
a
x
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
a
y
a
x
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-4 0
-2 0
0
20
40
-180 -90 0 90 180
θ (Degree)
A
... Therefore, one downstream network can extract or read out X target+ X gaze from the activity of parietal neurons, while another downstream network, using similar mechanisms, can read out Xtarget-X gaze, for instance, from the same responses, (Slinas & Abbott, 1995) (Salinas & Abbott, 1996). (Pouget & Sejnowski, 1997) Have elaborated on the above work and also have simulated a lesion in a model parietal neurons using a recurrent neural network(S Deneve, Latham, & Pouget, 2001), . They found that the model have reproduced many of the effects that are found in patients with similar lesion. ...
A Review of Neuroscience-Inspired Computer Vision: Deep Learning and Beyond
Preprint
Full-text available
  • Feb 2024
Our Study of gain modulation and active vision from the field of neuroscience has led us to define visual attention models or attention-based models in computer vision the following definition: Attention-based models are those in which a visual scene is processed part by part sequentially and not at once. In this paper, we review gain modulation, active vision and other neuroscience principles and see how knowledge gained from these fields has been utilized by computer vision. Also, we foresee what else need to be done in the future by professionals in computer vision in order to increase utilization of neuroscience to build more accurate visual models.
View
... It is possible to calculate the position of the stimulus in craniotopic coordinates from gain fields [23,24]. Two possibilities for the source of the eye position signal have been proposed: corollary discharge of an intended eye position signal such as the one found on the extraocular motor neurons [2], and from direct measurement of the eye position. ...
Mice require proprioception to establish long-term visuospatial memory
Preprint
  • Oct 2023
Because the retina moves constantly, the retinotopic representation of the visual world is spatially inaccurate and the brain must transform this spatially inaccurate retinal signal to a spatially accurate signal usable for perception and action. One of the salient discoveries of modern neuroscience is the role of the hippocampus in establishing gaze-independent, long-term visuospatial memories. The rat hippocampus has neurons which report the animal's position in space regardless of its angle of gaze. Rats with hippocampal lesions are unable to find the location of an escape platform hidden in a pool of opaque fluid, the Morris Water Maze (MWM) based on the visual aspects of their surrounding environment. Here we show that the representation of proprioception in the dysgranular zone of primary somatosensory cortex is equivalently necessary for mice to learn the location of the hidden platform, presumably because without it they cannot create a long-term gaze-independent visuospatial representation of their environment from the retinal signal. They have no trouble finding the platform when it is marked by a flag, and they have no motor or vestibular deficits.
View
... In neural networks, coordinate transforms amount to steerable neural mappings, projections from the original to the transformed representation that are steered by a parameter such as the direction of gaze relative to the body . Such mappings may be neurally implemented in what is known as gain fields (Pouget and Sejnowski 1997), essentially joint representations of the original and the steering dimension (Fig. 5). Peaks in the gain field form where input from the two sources overlaps. ...
Neural dynamic foundations of a theory of higher cognition: the case of grounding nested phrases
Article
Full-text available
  • Oct 2023
  • COGN NEURODYNAMICS
Because cognitive competences emerge in evolution and development from the sensory-motor domain, we seek a neural process account for higher cognition in which all representations are necessarily grounded in perception and action. The challenge is to understand how hallmarks of higher cognition, productivity, systematicity, and compositionality, may emerge from such a bottom-up approach. To address this challenge, we present key ideas from Dynamic Field Theory which postulates that neural populations are organized by recurrent connectivity to create stable localist representations. Dynamic instabilities enable the autonomous generation of sequences of mental states. The capacity to apply neural circuitry across broad sets of inputs that emulates the function call postulated in symbolic computation emerges through coordinate transforms implemented in neural gain fields. We show how binding localist neural representations through a shared index dimension enables conceptual structure, in which the interdependence among components of a representation is flexibly expressed. We demonstrate these principles in a neural dynamic architecture that represents and perceptually grounds nested relational and action phrases. Sequences of neural processing steps are generated autonomously to attentionally select the referenced objects and events in a manner that is sensitive to their interdependencies. This solves the problem of 2 and the massive binding problem in expressions such as “the small tree that is to the left of the lake which is to the left of the large tree”. We extend earlier work by incorporating new types of grammatical constructions and a larger vocabulary. We discuss the DFT framework relative to other neural process accounts of higher cognition and assess the scope and challenges of such neural theories.
View
... Importantly, although NU spatial tuning properties were generally inconsistent with transforming otolith signals towards body-centered coordinates, this does not necessarily rule out a role for these cells in the required transformations. In particular, theoretical studies have shown that cells at intermediate stages of neural networks involved in computing reference frame transformations often reflect posture-dependent changes in response gain (i.e., "gain fields"; Zipser and Andersen, 1988;Abbott, 1995, 1996;Pouget and Sejnowski, 1997;Xing and Andersen, 2000;Deneve et al., 2001;Blohm et al., 2009) . Thus, as a next step we also examined how cell response gains and phases in the cell's PD changed across head orientations (Fig. 6A). ...
Reference frames for encoding of translation and tilt in the caudal cerebellar vermis
Preprint
Full-text available
  • Sep 2023
Many daily behaviors rely critically on estimates of our body's motion and orientation in space. Vestibular signals are essential for such estimates but to contribute appropriately two key sets of computations are required. First, ambiguous motion information from the otolith organs must be combined with spatially transformed rotational signals (e.g., from the canals) to distinguish head translation from tilt. Second, tilt and translation estimates must be transformed from a head- to a body-centered reference frame to correctly interpret the body's motion. Studies have shown that cells in the caudal cerebellar vermis (nodulus and ventral uvula, NU) reflect the output of the first set of computations to estimate translation and tilt. However, it remains unknown whether these estimates are encoded exclusively in head-centered coordinates or whether they reflect a further transformation towards body-centered coordinates. Here we addressed this question by examining how the 3D spatial tuning of otolith and canal signals on translation- and tilt-selective NU neurons varies with changes in head-re-body and body-re-gravity orientation. We show that NU cell tuning properties are consistent with head-centered coding of otolith signals during translation. Furthermore, while canals signals in the NU have been transformed into world-referenced estimates of reorientation relative to gravity (i.e., tilt), as needed to resolve the tilt-translation ambiguity, the resulting tilt estimates are encoded in head-centered coordinates. Our results thus suggest that body-centered motion and orientation estimates required for postural control, navigation and reaching are computed elsewhere either by further transforming NU outputs or via computations in other parallel pathways.
View
View
View
Integration of Vestibular, Visual and Proprioceptive Inputs in the Cerebral Cortex during Movement Control
Article
  • Mar 2023
The review of the literature data is devoted to the integration of vestibular, visual and proprioceptive inputs in various areas of the cerebral cortex in humans and monkeys during movement control. Despite the abundance of studies of numerous areas of the cortex with vestibular and sensorimotor inputs, their functions and connections are insufficiently studied and understood. The review provides a relatively detailed analysis of data from recent studies of three areas of the cortex involved in motion control: region 7a of the posterior parietal cortex, in which responses to a combined visual-vestibular stimulus tended to dominate the vestibular input over the visual one; the visual region of the cingulate sulcus, which presumably integrates not only visual and vestibular afferent signals, but also proprioceptive signals from the lower limbs, thereby providing interaction between the sensory and motor systems during locomotion; and the area of the superior parietal lobule, in which the visual and somatic inputs interact, allowing you to control behavior when reaching and grasping an object. It is concluded that it is necessary to combine complex natural tasks with normative behavioral models in future research in order to understand how the brain converts sensory input data into a behavioral format.
View
View
Sympathetic Magic: A Psychological Enquiry
Article
  • Nov 2023
Sympathetic magic features strongly in virtually all religious traditions and in folk customs generally. Scholars agree that It is based on the association of ideas perceived as external, mind-independent causal realities, as connections mediating causal influence. Moreover, religious folk believe that this mediation involves forms of supernatural agency. From a psychological perspective, the key question revolves around the principles by which the cognitive system deems some of its content to reference the external world and other content to constitute internal mental forms of activity like thoughts, feelings and attitudes. The paper proposes that the critical factor has to do with the balance between two distinctly different kinds of cognitive content: representations of things (mentation arising in the form of something other than itself), as distinct from registrations referencing the intrinsic phenomenal properties of the mental state itself. The balance between these two determines whether content is perceived as external worldly event or a form of internal mental content.
View
Semi-orthogonal subspaces for value mediate a tradeoff between binding and generalization
Article
  • Sep 2023
When choosing between options, we must associate their values with the action needed to select them. We hypothesize that the brain solves this binding problem through neural population subspaces. To test this hypothesis, we examined neuronal responses in five reward-sensitive regions in macaques performing a risky choice task with sequential offers. Surprisingly, in all areas, the neural population encoded the values of offers presented on the left and right in distinct subspaces. We show that the encoding we observe is sufficient to bind the values of the offers to their respective positions in space while preserving abstract value information, which may be important for rapid learning and generalization to novel contexts. Moreover, after both offers have been presented, all areas encode the value of the first and second offers in orthogonal subspaces. In this case as well, the orthogonalization provides binding. Our binding-by-subspace hypothesis makes two novel predictions borne out by the data. First, behavioral errors should correlate with putative spatial (but not temporal) misbinding in the neural representation. Second, the specific representational geometry that we observe across animals also indicates that behavioral errors should increase when offers have low or high values, compared to when they have medium values, even when controlling for value difference. Together, these results support the idea that the brain makes use of semi-orthogonal subspaces to bind features together.
View
Object-Centered Neglect in Patients with Unilateral Neglect: Effects of Left-Right Coordinates of Objects
Article
Full-text available
  • Dec 1994
Abstract When patients with right-sided hemispheric lesions neglect information on the left side, with respect to what set of spatial coordinates is left defined? Two potential reference frames were examined in this study, one where left and right are defined with respect to the midline of the viewer and/or environment (viewer/env-centered) and the other where left and right are defined with respect to the midline of the object (object-centered). By rotating the stimulus 90° clockwise or counterclockwise, and instructing patients with neglect to report the colors appearing around the border of a stimulus, an independent measure was obtained for the number of colors reported from the left and right of the viewer/env- and from the object-based reference frame. Whereas significant object-centered neglect was observed only for upper case asymmetrical letters but not for symmetrical letters nor for drawings of familiar animals or objects, significant viewer/env-based neglect was observed with all the stimulus types. We present an account of the coexistence of neglect in more than one frame of reference and the presence of object-centered neglect under a restricted set of conditions.
View
View
View
Spatial analysis of somatosensory and visual stimuli by single neurons in macaque area 7B
Article
  • Jan 1996
Objects detected through different sensory modalities are perceived as having positions relative to a common spatial framework. To study the neural substrates of this phenomenon, we have recorded from neurons in area 7B of alert macaque monkeys, using a paradigm in which/ by varying gaze-angle and arm position/ we dissociate the proximal coordinates of a stimulus(as referred to the retina or to the skin surface of the hand) from its distal coordinates (as referred to extrapersonal space).Some neurons sensitive to stimulation of the hand respond more or less strongly as a function of arm position. Some neurons responsive to visual stimulation are also influenced by the angle of gaze. We conclude that(a) neurons in separate divisions of area 7B contribute to spatial preprocessing of visual and hand-somatosensory stimuli and(b) that full integration of visual and hand-somatosensory inputs/ at the single neuron level must occur at sites downstream from area 7B. McDonnel1-Pew Program in Cognitive Neuroscience Individual Research Grant 90-37 to CRO.
View
Distributed Hierarchical Processing in the Primate Cerebral Cortex
Article
  • Jan 1991
In recent years, many new cortical areas have been identified in the macaque monkey. The number of identified connections between areas has increased even more dramatically. We report here on (1) a summary of the layout of cortical areas associated with vision and with other modalities, (2) a computerized database for storing and representing large amounts of information on connectivity patterns, and (3) the application of these data to the analysis of hierarchical organization of the cerebral cortex. Our analysis concentrates on the visual system, which includes 25 neocortical areas that are predominantly or exclusively visual in function, plus an additional 7 areas that we regard as visual-association areas on the basis of their extensive visual inputs. A total of 305 connections among these 32 visual and visual-association areas have been reported. This represents 31% of the possible number of pathways it each area were connected with all others. The actual degree of connectivity is likely to be closer to 40%. The great majority of pathways involve reciprocal connections between areas. There are also extensive connections with cortical areas outside the visual system proper, including the somatosensory cortex, as well as neocortical, transitional, and archicortical regions in the temporal and frontal lobes. In the somatosensory/motor system, there are 62 identified pathways linking 13 cortical areas, suggesting an overall connectivity of about 40%. Based on the laminar patterns of connections between areas, we propose a hierarchy of visual areas and of somato sensory/motor areas that is more comprehensive than those suggested in other recent studies. The current version of the visual hierarchy includes 10 levels of cortical processing. Altogether, it contains 14 levels if one includes the retina and lateral geniculate nucleus at the bottom as well as the entorhinal cortex and hippocampus at the top. Within this hierarchy, there are multiple, intertwined processing streams, which, at a low level, are related to the compartmental organization of areas V1 and V2 and, at a high level, are related to the distinction between processing centers in the temporal and parietal lobes. However, there are some pathways and relationships (about 10% of the total) whose descriptions do not fit cleanly into this hierarchical scheme for one reason or another. In most instances, though, it is unclear whether these represent genuine exceptions to a strict hierarchy rather than inaccuracies or uncertainties in the reported assignment.
View
View
View
Sensori-motor integration in the primate superior colliculus
Article
  • Feb 1991
  • Semin Neurosci
The sudden onset of a novel stimulus usually triggers orienting responses of the eyes, head and external ears (pinnae). These responses facilitate the reception of additional signals originating from the source of the stimulus and assist in the sensory guidance of appropriate limb and body movements. A midbrain structure, the superior colliculus, plays a critical role in triggering and organizing orienting movements and is a particularly interesting structure for studying the neural computations involved in the translation of sensory signals into motor commands. Auditory, somatosensory and visual signals converge in its deep layers, where neurons are found that generate motor commands for eye, head and pinna movements. This article focuses on the role of the superior colliculus in the control of saccadic (quick, high-velocity) eye movements with particular regard to three issues related to the functional properties of collicular neurons. First, how do neurons with large movement fields specify accurately the direction and amplitude of an eye movement? Second, how are signals converted from different sensory modalities into commands in a common motor frame of reference? Last, how are the motor command signals found in the superior colliculus transformed into those needed by the motor neuron pools innervating the extraocular muscles?
View
View
View