Content uploaded by Barrie Frost
Author content
All content in this area was uploaded by Barrie Frost
Content may be subject to copyright.
A preview of the PDF is not available
Computation of different optical variables of looming objects in pigeon nucleus rotundus neurons
Abstract
Three types of looming-selective neurons have been found in the nucleus rotundus of pigeons, each computing a different optical variable related to image expansion of objects approaching on a direct collision course with the bird. None of these neurons respond to simulated approach toward stationary objects. A detailed analysis of these neurons' firing pattern to approaching objects of different sizes and velocities shows that one group of neurons signals relative rate of expansion tau (tau), a second group signals absolute rate of expansion rho (rho), and a third group signals yet another optical variable eta (eta). The rho parameter is required for the computation of both tau and eta, whose respective ecological functions probably provide precise 'time-to-collision' information and 'early warning' detection for large approaching objects.
... Detecting objects approaching on a collision course is critical for the survival of animals in the environment. Specialized neurons or brain circuits highly sensitive to looming stimuli have been identified in many species, including insects [1], fish [2], pigeons [3], mice [4,5], and others. In primates, both adult monkeys and human infants elicit avoidance behaviors to symmetrically looming stimuli [6,7], imaging studies mainly revealed looming sensitive responses in the cortical brain regions [8][9][10][11][12]. ...
... In the optic tectum and the downstream nucleus rotundus (homologues of the SC and pulvinar in mammals) of pigeons, different types of neurons encode several optical variables of a looming stimulus, including the time-to-collision, absolute rate of expansion, and object size [3]. In the mouse SC, Shang and colleagues identified parvalbumin-positive (PV+) excitatory projection neurons in the superficial layers encoding the optical parameters of a looming stimulus in their receptive fields [4,16]. ...
Detecting imminent collisions is essential for survival. Here, we used high-resolution fMRI at 7 Tesla to investigate the role of attention and consciousness for detecting collision trajectory in human subcortical pathways. Healthy participants can precisely discriminate collision from near-miss trajectory of an approaching object, with pupil size change reflecting collision sensitivity. Subcortical pathways from the superior colliculus (SC) to the ventromedial pulvinar (vmPul) and ventral tegmental area (VTA) exhibited collision-sensitive responses even when participants were not paying attention to the looming stimuli. For hemianopic patients with unilateral lesions of the geniculostriate pathway, the ipsilesional SC and VTA showed significant activation to collision stimuli in their scotoma. Furthermore, stronger SC responses predicted better behavioral performance in collision detection even in the absence of awareness. Therefore, human tectofugal pathways could automatically detect collision trajectories without the observers’ attention to and awareness of looming stimuli, supporting “blindsight” detection of impending visual threats.
... For visual detection of impending threats, diverse animal groups, including mammals, fish, insects, crustaceans and birds possess neurons that specifically respond to visual looming stimuli, i.e. objects approaching on a collision course with constant velocity (Bennett et al., 2019;Bhattacharyya et al., 2017;de Vries & Clandinin, 2012;Dunn et al., 2016;Fotowat et al., 2009;Liu et al., 2011;Nakagawa & Hongjian, 2010;Oliva & Tomsic, 2014;Preuss et al., 2006;Sun & Frost, 1998;Temizer et al., 2015;Wu et al., 2005). These studies showed that the evoked escape behaviour to visual looms differs between species, while response timing is often, but not always, comparable (Branco & Redgrave, 2020;Hemmi & Tomsic, 2012). ...
... The peak firing rate of these neurons comes a fixed neural delay after the looming stimulus reaches a threshold angular size, independent of the stimulus parameter γ (or equivalently, of its approach speed assuming a fixed half-size, l). A phenomenological model, called the η model, predicts this key aspect of their firing rate (Gabbiani et al., 1999;Sun & Frost, 1998). In this model, the time-dependent firing rate is (up to a scaling factor) obtained by multiplying the angular speed of the object during approach by a non-linear function of its angular size: ...
In animal species ranging from invertebrate to mammals, visually guided escape behaviours have been studied using looming stimuli, the two‐dimensional expanding projection on a screen of an object approaching on a collision course at constant speed. The peak firing rate or membrane potential of neurons responding to looming stimuli often tracks a fixed threshold angular size of the approaching stimulus that contributes to the triggering of escape behaviours. To study whether this result holds more generally, we designed stimuli that simulate acceleration or deceleration over the course of object approach on a collision course. Under these conditions, we found that the angular threshold conveyed by collision detecting neurons in grasshoppers was sensitive to acceleration whereas the triggering of escape behaviours was less so. In contrast, neurons in goldfish identified through the characteristic features of the escape behaviours they trigger, showed little sensitivity to acceleration. This closely mirrored a broader lack of sensitivity to acceleration of the goldfish escape behaviour. Thus, although the sensory coding of simulated colliding stimuli with non‐zero acceleration probably differs in grasshoppers and goldfish, the triggering of escape behaviours converges towards similar characteristics. Approaching stimuli with non‐zero acceleration may help refine our understanding of neural computations underlying escape behaviours in a broad range of animal species. image
Key points
A companion manuscript showed that two mathematical models of collision‐detecting neurons in grasshoppers and goldfish make distinct predictions for the timing of their responses to simulated objects approaching on a collision course with non‐zero acceleration.
Testing these experimental predictions showed that grasshopper neurons are sensitive to acceleration while goldfish neurons are not, in agreement with the distinct models proposed previously in these species using constant velocity approaches.
Grasshopper and goldfish escape behaviours occurred after the stimulus reached a fixed angular size insensitive to acceleration, suggesting further downstream processing in grasshopper motor circuits to match what was observed in goldfish.
Thus, in spite of different sensory processing in the two species, escape behaviours converge towards similar solutions.
The use of object acceleration during approach on a collision course may help better understand the neural computations implemented for collision avoidance in a broad range of species.
... Situations like the one illustrated by this example are already getting a good deal of experimental support. Tau (τ) has been used not only as a part of artificial models of motor control (e.g., de Rugy et al., 2002), but it has also been found in the brain activity of primates while executing different tasks (Georgopoulos, 2007;Merchant & Georgopoulos, 2006;Merchant et al., 2004Merchant et al., , 2003aMerchant et al., , 2003bPort et al., 2001), in the nucleus rotundus of pigeons (Sun & Frost, 1998;Wang & Frost, 1992), in babies' brains while performing a visual attention task (van der Weel & van der Meer, 2009), or in the brain of adult humans while executing interception tasks (van der Weel et al., 2022), time-to-collision tasks (Field & Wann, 2005), or just simple laboratory tasks (Tan et al., 2009). All these studies, along with other recent involving resonance, ecological information and artificial neural networks (e.g., Falandays et al., 2023;Hasson et al., 2020) show the feasibility of the experimental implementation of the motif of resonance within different fields of the neurosciences. ...
... For the retinotopic ring stimuli, we calculated timecourses using t and h based on the visual angle parameters at which the videos were presented iScience Article to participants, using formulas from. 92 We fit this optical variable encoding model, along with several variations using different combinations of predictors (Table S1), using the same method described above. ...
The neural computations for looming detection are strikingly similar across species. In mammals, information about approaching threats is conveyed from the retina to the midbrain superior colliculus, where approach variables are computed to enable defensive behavior. Although neuroscientific theories posit that midbrain representations contribute to emotion through connectivity with distributed brain systems, it remains unknown whether a computational system for looming detection can predict both defensive behavior and phenomenal experience in humans. Here, we show that a shallow convolutional neural network based on the Drosophila visual system predicts defensive blinking to looming objects in infants and superior colliculus responses to optical expansion in adults. Further, the neural network’s responses to naturalistic video clips predict self-reported emotion largely by way of subjective arousal. These findings illustrate how a simple neural network architecture optimized for a species-general task relevant for survival explains motor and experiential components of human emotion.
... Over the years, a long standing debate has animated the literature about whether interceptive actions could be afforded entirely by optical variables derived directly from available visual signals, such as the ratio (τ) of the object's image retinal size and its expansion rate originally proposed by Hoyle (1957) and Weinberger (1971), and reinvigorated by Lee (1976), or the distance between the approaching object and the observer (Collewijn, 1972;Carl and Gellman, 1987;Gellman and Carl, 1991;van Donkelaar et al., 1992;Port et al., 1997). In particular, Lee's τ model, inspired by Gibson's ecological approach (Lee, 1976;Gibson, 1979;Lee and Reddish, 1981), motivated much research in the manual interception field, receiving support from psychophysical work that applied and revisited the model to explain interceptive behavior in various experimental conditions (Bootsma and Oudejans, 1993;Peper et al., 1994;Rushton and Wann, 1999), as well as from neurophysiological evidence that neurons in the optic tectum of the pigeon may encode time-to-contact information in line with the τ model predictions (Sun and Frost, 1998). ...
Introduction: Recent views posit that precise control of the interceptive timing can be achieved by combining on-line processing of visual information with predictions based on prior experience. Indeed, for interception of free-falling objects under gravity’s effects, experimental evidence shows that time-to-contact predictions can be derived from an internal gravity representation in the vestibular cortex. However, whether the internal gravity model is fully engaged at the target motion outset or reinforced by visual motion processing at later stages of motion is not yet clear. Moreover, there is no conclusive evidence about the relative contribution of internalized gravity and optical information in determining the time-to-contact estimates.
Methods: We sought to gain insight on this issue by asking 32 participants to intercept free falling objects approaching directly from above in virtual reality. Object motion had durations comprised between 800 and 1100 ms and it could be either congruent with gravity (1 g accelerated motion) or not (constant velocity or -1 g decelerated motion). We analyzed accuracy and precision of the interceptive responses, and fitted them to Bayesian regression models, which included predictors related to the recruitment of a priori gravity information at different times during the target motion, as well as based on available optical information.
Results: Consistent with the use of internalized gravity information, interception accuracy and precision were significantly higher with 1 g motion. Moreover, Bayesian regression indicated that interceptive responses were predicted very closely by assuming engagement of the gravity prior 450 ms after the motion onset, and that adding a predictor related to on-line processing of optical information improved only slightly the model predictive power.
Discussion: Thus, engagement of a priori gravity information depended critically on the processing of the first 450 ms of visual motion information, exerting a predominant influence on the interceptive timing, compared to continuously available optical information. Finally, these results may support a parallel processing scheme for the control of interceptive timing.
... Other studies have since demonstrated the use of binocular optic flow to control velocity during forward flight (Schiffner and Srinivasan 2015) and position during hovering flight (Goller and Altshuler 2014). A1 of the OCb also receives, through PL, binocular inputs from the layer 13 of the optic tectum and descending inputs from the tectofugal pathways, both of which have been shown to be involved in the detection of looming (Sun and Frost 1998;Wu et al. 2005;Xiao et al. 2006). Looming is a signal that relates to time to collision, which is useful for rapid escape, feeding and landing (Wang and Frost 1992;Lee 2009). ...
Avian flight is guided by optic flow—the movement across the retina of images of surfaces and edges in the environment due to self-motion. In all vertebrates, there is a short pathway for optic flow information to reach pre-motor areas: retinal-recipient regions in the midbrain encode optic flow, which is then sent to the cerebellum. One well-known role for optic flow pathways to the cerebellum is the control of stabilizing eye movements (the optokinetic response). However, the role of this pathway in controlling locomotion is less well understood. Electrophysiological and tract tracing studies are revealing the functional connectivity of a more elaborate circuit through the avian cerebellum, which integrates optic flow with other sensory signals. Here we review the research supporting this framework and identify the cerebellar output centres, the lateral (CbL) and medial (CbM) cerebellar nuclei, as two key nodes with potentially distinct roles in flight control. The CbM receives bilateral optic flow information and projects to sites in the brainstem that suggest a primary role for flight control over time, such as during forward flight. The CbL receives monocular optic flow and other types of visual information. This site provides feedback to sensory areas throughout the brain and has a strong projection the nucleus ruber, which is known to have a dominant role in forelimb muscle control. This arrangement suggests primary roles for the CbL in the control of wing morphing and for rapid maneuvers.
Ecological psychology is one of the main alternative theories of perception and action available in the contemporary literature. This Element explores and analyzes its most relevant ideas, concepts, methods, and experimental results. It discusses the historical roots of the ecological approach. The Element then analyzes the works of the two main founders of ecological psychology: James and Eleanor Gibson. It also explores the development of ecological psychology since the 1980s until nowadays. Finally, the Element identifies and evaluates the future of the ecological approach to perception and action.
Saccades are rapid eye movements that are used by all species with good vision. They have been extensively studied, especially in vertebrates, and are understood to be controlled by a conserved brainstem circuit. However, despite the fact that saccades play important roles during diverse visually guided behaviours, little is known about whether their properties, including the manner in which they are coordinated with head/body movements, vary in the context of different visuomotor tasks. Here, we characterise the saccadic repertoire of larval zebrafish and identify five saccade types, defined by systematic differences in kinematics and binocular coordination. Each type was differentially expressed during visually guided behaviours. Conjugate saccades form a large group that are used in at least four contexts: Fast phases of the optokinetic nystagmus, visual scanning in stationary animals, and to shift or maintain gaze during locomotion. Convergent saccades play a specialised role during hunting and are coordinated with body movements to foveate prey. Furthermore, conjugate and convergent saccades follow distinct velocity main sequence relationships and show differences in the millisecond coordination of the eyes and body, pointing to differences in underlying neurophysiology. In summary, this study reveals unexpected diversity in horizontal saccades and predicts saccade type-specific neural activity patterns.
Highlights
Kinematic analysis of thousands of rapid eye movements reveals five saccade types.
Conjugate saccades have at least four identifiable visual functions.
Convergent saccades are coordinated with body movements to foveate prey.
Timing, kinematics and main sequence relationships indicate saccade type-specific neural control.
Many creatures with small brains and simple nervous systems - such as insects - are astonishingly good at coping with the world around them. A fly, for example, can deftly evade a swat, manoeuvre perfectly well in a cluttered world, and execute a flawless landing on the rim of a teacup. Do such creatures use clever short-cuts to vision and navigation, and if so, can these tricks be exploited to create new kinds of robots? These questions are explored in this book, which contains articles by experimental biologists as well as computer scientists, in this newly emerging multidisciplinary field. This is a fresh approach to an area of research that has traditionally been dominated by engineering methods, and the book is written in a style in which technical jargon is kept to a minimum.
Studies of sensitivity of striate cortical neurones to textured, as well as to oriented, stimuli in the lightly anaesthetised eat present a picture of cortical processing strikingly different from classical views of purely serial (hierarchical) processing by cells acting as “feature detectors”. The results indicate a strong element of parallel processing by simple and complex cortical cells and significant influences of moving textured backgrounds on sensitivity to foreground objects.
Purpose. To develop quantitative models to explain the physiological response properties of looming sensitive neurons in nucleus rotundus (nRt) that signal impending collision of an approaching object (see the companion abstract). Methods. These models were created based on the physiological responses (both qualitative and quantitative data) recorded to various stimulus conditions (including direct manipulation of various optical variables that could be specified by a looming object). These models also take into account the physiological response properties and anatomical connection of the optic tectum that sends a major input to nRt. Results. For these Rt looming detectors, the receptive field (RF) could be composed of a radial arrangement of concentric arrays of RFs of simple local motion detectors (possibly tectal neurons), with the centre of expansion overlapping with the centre of RF radial layout. These tectal neurons would prefer movement directions that are oriented radially from the centre of the concentric array and they then converge onto Rt looming detectors. Based on this basic RF organization, a series of quantitative models were formulated to account for the specific properties of the time course of Rt neurons response to impending collision. Specifically it is shown how the ratio of the angular velocity over the visual angle subtense, extracted from the approaching object, could be easily calculated. Such a ratio (equal to I/T) provides information about the time-to-collision with the object (Lee, 1980). Concisions. These neuronal models provide explanations for the various physiological responses found in different classes of Rt looming detectors. Not only are they potentially informative for understanding the neuronal coding of motion in depth, but these models could provide important insights for robotics and machine vision.
Contemporary concepts of the organization of the visual system began to emerge in the mid to late 19th century. It had long been appreciated that penetrating wounds of the occipital region of the brain resulted in blindness. Lesions of more lateral portions of the hemisphere often resulted in varying degrees of visual agnosias, though it was not until the middle third of the twentieth century that the contribution of such cortical areas to visual performance became an object of interest.
The visual world of most animals typically presents stimuli in the context of surrounding stimuli, a situation which contrasts sharply with the normal practice in most vision laboratories. Yet, countless studies of human visual perception have shown how the contexual relationship in which a stimulus occurs can dramatically alter our perception of local visual events. For example, for many of the fundamental dimensions of visual stimuli, such as size (34, 75, 76), brightness (22, 37), hue (52), orientation (18, 101), and motion (25, 58, 59, 60, 127), the context provided by the surrounding visual field has been shown to exert considerable influence on their appearance. In fact, all of the many forms of contrast illusion, such as simultaneous colour contrast, brightness contrast, velocity contrast etc., exploit these effects. It has also been realised for a considerable time by students of perception, that this type of “perceptual relativity” plays a central role in producing the perceptual constancies (53). Indeed, presentation of stimuli in isolation is one of the classic perceptual manipulations used to destroy perceptual constancy effects (54, 120). Over the past few years we have been attempting to describe the processing that goes on in certain regions of the visual system of pigeons and cats with respect to the stimulus context manipulation.
Due to recent behavioral and electrophysiological data found in different anurans, some investigators believe that the visual system of frogs and toads is a highly specialized machinery which detects only self-moving visual signals relevant to the survival of the animals (p. 357 f., 435 ff.). Other visual signals are believed to be “suppressed” by the neuronal network of the visual system. Thus the ironic poem of Heinrich Heine would be incorrect as such a neuronal machine leaves little possibility for frogs to “erquicken… an Sonnenblicken”. The angular velocity of the sun and the shadows cast by stationary objects in the frog’s habitat would be too slow to be discovered by the movement-detecting neuronal systems.
Birds share with many other animal species a highly mobile life-style which produces, amongst other things, a very rich diversity of patterns of motion across their visual fields. As we have argued elsewhere (Frost 1982, 1985, 1993; Frost et al. 1990), these different patterns of visual motion fall generally into two broad classes, object motion and self-induced motion, the former usually resulting from some action by another animal, while the latter is usually produced by some action of the observing animal itself. Each category contains a large number of ecologically specific patterns of visual motion and, in a natural environment, patterns from one or both categories may occur concurrently. It is the task of the visual system to segregate or parse these various patterns of motion into the appropriate events that gave rise to them, and integrate them with other visual features, so that ultimately the bird may respond with behaviour that is appropriate for the situation.