Fig 4 - uploaded by Alexandre Pitti
Content may be subject to copyright.
Mechanism of STDP with 0A = A = +1and = = 20 ms. (a) Each time a post-synaptic neuron fires, its synaptic weights are decreased by A , and each time a synapse receives an action potential, its synaptic weight is incremented by an amount A. (b) Based on this mechanism, different neural pairs can assemble themselves into asynchronous neuronal groups (polychronized groups; see [23]).
Source publication
Agency is the sense that I am the cause or author of a movement. Babies develop early this feeling by perceiving the contingency between afferent (sensor) and efferent (motor) information. A comparator model is hypothesized to be associated with many brain regions to monitor and simulate the concordance between self-produced actions and their conse...
Contexts in source publication
Context 1
... neurons (see [19]- [21]). They are signifi- cant mechanisms for both activity-dependent development of neural circuitry and adult memory storage. The time delay between the presynaptic neuron spiking and the postsynaptic neuron firing corresponds to the interval range of activation of their synaptic plasticity and weight adaptation ; see Fig. 4 ...
Context 2
... pairs can be viewed as small conditional scripts, which can detect/encode the contingency at the local level if neuron fires at time then neuron fires at time This mechanism, although simple at the neurons' scale, can generate very complex dynamics as the neural pairs can aggre- gate themselves into long-range spatiotemporal clusters [e.g., Fig. 4(b)]; see [42] and [43]. In sensorimotor networks, we propose that these assembled spatiotemporal patterns constitute a repertoire of commands or action primitives as Wolpert con- ceives them (see [25] and [43]). They represent internal models, which are the building blocks used to construct intricate motor behaviors with an enormous ...
Context 3
... in front of its own reflection. The neural dynamics in the vision map re- veal that an entrainment effect has occurred with a strong syn- chrony. As expected, the motion information is very salient and perfectly contingent to its own action (i.e., proprioceptive in- formation), which makes the mirror perceptual experience very unique. We plot in Fig. 14 its corresponding agency index. Com- pared with the normal situation in Fig. 9, the agency index jumps Fig. 12. Mirror experience. When our head-like robot scrutinizes its own reflection in front of the mirror [camera view in b)], most of the salient information gets centered in the middle of the scene, which is not the case in normal ...
Context 4
... that one person's motion or its own reflection induces more saliency in terms of senso- rimotor conflicts and coordinations rather than for objects and visual scenes. The embodied system, in a way, modulates and combines its own agency with those of other people, sharing the same circuits. Hence, rather than strict self-other distinction, the Fig. 14. Agency index in front of a mirror (see Fig. 13). Soon after presenting the mirror in front of the robot, the agency index rise too a very high peak above the normal situation of live enaction. Fig. 15. Agency index in front of another person. Facing a person produces values similar to the mirror experience but higher values compared ...
Similar publications
This investigation of the leech heartbeat neural network system led to the development of a low resources, real-time, biomimetic digital hardware for use in hybrid experiments. The leech heartbeat neural network is one of the simplest central pattern generators (CPG). In biology, CPG provide the rhythmic bursts of spikes that form the basis for all...
Neural signals for locomotion are influenced both by the neural network architecture and sensory inputs coordinating and adapting the gait to the environment. Adaptation relies on the ability to change amplitude, frequency, and phase of the signals within the sensorimotor loop in response to external stimuli. However, in order to experiment with cl...
Spiking Neural Network (SNN), as a brain-inspired machine learning algorithm, is closer to the computing mechanism of human brain and more suitable to reveal the essence of intelligence compared with Artificial Neural Networks (ANN), attracting more and more attention in recent years. In addition, the information processed by SNN is in the form of...
Existing methods in neuromorphic olfaction mainly focus on implementing the data transformation based on the neurobiological architecture of the olfactory pathway. While the transformation is pivotal for the sparse spike-based representation of odor data, classification techniques based on the bio-computations of the higher brain areas, which proce...
![Figure 1: Simple Hardware Trojan, adapted from [12].](profile/Kyle-Madden-3/publication/329655963/figure/fig1/AS:703756245159937@1544799972765/Simple-Hardware-Trojan-adapted-from-12_Q320.jpg)
![Figure 2: 3x3 Mesh network with assorted resources, adapted from [14].](profile/Kyle-Madden-3/publication/329655963/figure/fig2/AS:703756245159938@1544799972787/3x3-Mesh-network-with-assorted-resources-adapted-from-14_Q320.jpg)
Modern computing systems are using Networks-on-Chip (NoCs) for scalable on-chip communications. Traditional security attacks have focused on the computing cores however modern attacks can focus on the NoC interconnect as a means to injecting unwanted traffic. This poses a significant security risk to physical systems that rely on sensory feedback a...
Citations
... Some disadvantages lie on the level of noise and the variability of the input dynamics in spiking recurrent neural networks, making the learning and control of a large number of clusters difficult, in comparison to supervised learning methods. This kind of spiking recurrent networks have been already evaluated in similar research on self-perception in visuo-motor control [23], and on visuo-tactile integration [24,25]. ...
We propose a developmental model inspired by the cortico-basal system (CX-BG) for vocal learning in babies and for solving the correspondence mismatch problem they face when they hear unfamiliar voices, with different tones and pitches. This model is based on the neural architecture INFERNO standing for Iterative Free-Energy Optimization of Recurrent Neural Networks. Free-energy minimization is used for rapidly exploring, selecting and learning the optimal choices of actions to perform (eg sound production) in order to reproduce and control as accurately as possible the spike trains representing desired perceptions (eg sound categories). We detail in this paper the CX-BG system responsible for linking causally the sound and motor primitives at the order of a few milliseconds. Two experiments performed with a small and a large audio database show the capabilities of exploration, generalization and robustness to noise of our neural architecture in retrieving audio primitives during vocal learning and during acoustic matching with unheared voices (different genders and tones).
... Several groups of researchers attempted to equip arti¯cial systems with this prospective sense of agency or closely related capacities. Pitti et al., endowed a robot with the ability to detect and À À À within \the here and now" [Pitti et al., 2009] À À À predict contingencies in sensorimotor networks using a neural network emulating spike timing-dependent synaptic plasticity, with the robot starting to act upon the contingencies. The authors took this as behavioral representation of one of the most basic levels of self-awareness and agency. ...
For humans, phenomenal experiences take up a central role in their daily interaction with the world. In this paper, we argue in favor of shifting phenomenal experiences into the focus of cognitive systems research and development. Instead of aiming to make artificial systems feel in the same way humans do, we focus on the possibilities of engineering capacities that are functionally equivalent to phenomenal experiences. These capacities can provide a different quality of input, enabling a cognitive system to self-evaluate its state in the world more effectively and with more generality than current methods allow. We ground our general argument using the example of the sense of agency. At the same time, we reflect on the broader possibilities and benefits for artificial counterparts to human phenomenal experiences and provide suggestions regarding the implementation of functionally equivalent mechanisms.
... All these exploited the spatio-temporal contingency, related to the sense of agency. Pitti et al. [53] studied temporal contingency perception and agency measure using spiking neural networks. Gain-field networks were employed to simultaneously learn reaching and body "self-perception" in [1]. ...
Self-recognition or self-awareness is a capacity attributed typically only to humans and few other species. The definitions of these concepts vary and little is known about the mechanisms behind them. However, there is a Turing test-like benchmark: the mirror self-recognition, which consists in covertly putting a mark on the face of the tested subject, placing her in front of a mirror, and observing the reactions. In this work, first, we provide a mechanistic decomposition, or process model, of what components are required to pass this test. Based on these, we provide suggestions for empirical research. In particular, in our view, the way the infants or animals reach for the mark should be studied in detail. Second, we develop a model to enable the humanoid robot Nao to pass the test. The core of our technical contribution is learning the appearance representation and visual novelty detection by means of learning the generative model of the face with deep auto-encoders and exploiting the prediction error. The mark is identified as a salient region on the face and reaching action is triggered, relying on a previously learned mapping to arm joint angles. The architecture is tested on two robots with completely different face.
... All these exploited the spatio-temporal contingency, related to the sense of agency. Pitti et al. [53] studied temporal contingency perception and agency measure using spiking neural networks. Gain-field networks were employed to simultaneously learn reaching and body "self-perception" in [1]. ...
Self-recognition or self-awareness is a capacity attributed typically only to humans and few other species. The definitions of these concepts vary and little is known about the mechanisms behind them. However, there is a Turing test-like benchmark: the mirror self-recognition, which consists in covertly putting a mark on the face of the tested subject, placing her in front of a mirror, and observing the reactions. In this work, first, we provide a mechanistic decomposition, or process model, of what components are required to pass this test. Based on these, we provide suggestions for empirical research. In particular, in our view, the way the infants or animals reach for the mark should be studied in detail. Second, we develop a model to enable the humanoid robot Nao to pass the test. The core of our technical contribution is learning the appearance representation and visual novelty detection by means of learning the generative model of the face with deep auto-encoders and exploiting the prediction error. The mark is identified as a salient region on the face and reaching action is triggered, relying on a previously learned mapping to arm joint angles. The architecture is tested on two robots with a completely different face.
... At present, SNNs have been well implemented in some brain regions modeling and cognitive functions simulation, like image classification (Zhang et al., 2018b), working memory maintenance (Zhang et al., 2016), decision-making tasks (Héricé et al., 2016;Zhao et al., 2018), cortical development (Khalil et al., 2017a), contingency perception (Pitti et al., 2009) etc. However, even though the training methods proposed by Zhang et al. (2018a) and Shrestha and Orchard (2018) make SNNs performance comparable to ANNs, they are at the cost of a large amount of time. ...
Spiking Neural Networks (SNNs) have shown favorable performance recently. Nonetheless, the time-consuming computation on neuron level and complex optimization limit their real-time application. Curiosity has shown great performance in brain learning, which helps biological brains grasp new knowledge efficiently and actively. Inspired by this leaning mechanism, we propose a curiosity-based SNN (CBSNN) model, which contains four main learning processes. Firstly, the network is trained with biologically plausible plasticity principles to get the novelty estimations of all samples in only one epoch; secondly, the CBSNN begins to repeatedly learn the samples whose novelty estimations exceed the novelty threshold and dynamically update the novelty estimations of samples according to the learning results in five epochs; thirdly, in order to avoid the overfitting of the novel samples and forgetting of the learned samples, CBSNN retrains all samples in one epoch; finally, step two and step three are periodically taken until network convergence. Compared with the state-of-the-art Voltage-driven Plasticity-centric SNN (VPSNN) under standard architecture, our model achieves a higher accuracy of 98.55% with only 54.95% of its computation cost on the MNIST hand-written digit recognition dataset. Similar conclusion can also be found out in other datasets, i.e., Iris, NETtalk, Fashion-MNIST, and CIFAR-10, respectively. More experiments and analysis further prove that such curiosity-based learning theory is helpful in improving the efficiency of SNNs. As far as we know, this is the first practical combination of the curiosity mechanism and SNN, and these improvements will make the realistic application of SNNs possible on more specific tasks within the von Neumann framework.
... The capacity we are discussing is linked to causality and can be detected as the difference between the probability that a certain outcome happens after a certain action is performed and the probability that the outcome happens if the action is not performed (Gopnik et al., 2004). To check the effectiveness of actions, the architecture discussed above should be enhanced with suitable mechanisms to observe the environment multiple times after having executed an action or not, and to compare the probability of achieving the outcome in the two cases (for some computational models using mechanisms to face the problem of agency see: Pitti et al., 2009;Butko and Movellan, 2010;Sperati and Baldassarre, 2017). ...
Much current work in robotics focuses on the development of robots capable of autonomous unsupervised learning. An essential prerequisite for such learning to be possible is that the agent should be sensitive to the link between its actions and the consequences of its actions, called sensorimotor contingencies. This sensitivity, and more particularly its role as a key drive of development, has been widely studied by developmental psychologists. However, the results of these studies may not necessarily be accessible or intelligible to roboticians. In this paper, we review the main experimental data demonstrating the role of sensitivity to sensorimotor contingencies in infants' acquisition of four fundamental motor and cognitive abilities: body knowledge, memory, generalization, and goal-directedness. We relate this data from developmental psychology to work in robotics, highlighting the links between these two domains of research. In the last part of the article we present a blueprint architecture demonstrating how exploitation of sensitivity to sensorimotor contingencies, combined with the notion of "goal," allows an agent to develop new sensorimotor skills. This architecture can be used to guide the design of specific computational models, and also to possibly envisage new empirical experiments.
... The capacity we are discussing is linked to causality and can be detected as the difference between the probability that a certain outcome happens after a certain action is performed and the probability that the outcome happens if the action is not performed (Gopnik et al., 2004). To check the effectiveness of actions, the architecture discussed above should be enhanced with suitable mechanisms to observe the environment multiple times after having executed an action or not, and to compare the probability of achieving the outcome in the two cases (for some computational models of this, see: Butko & Movellan, 2010;Pitti et al., 2009;Sperati & Baldassarre, 2017). ...
Au regard de la place croissante occupée par l'intelligence artificielle dans nos sociétés, la nécessité d'affiner notre compréhension de la notion d'apprentissage semble plus importante que jamais. S'intégrant dans une telle optique, cette thèse de doctorat cherche à mettre en lumière les mécanismes d'apprentissage permettant au bébé d'acquérir au cours de la première année de vie une utilisation appropriée et différenciée de son corps lui permettant d'interagir de manière efficace avec son environnement physique et social, ce que nous désignons ici par le terme de "savoir-fairecorporel". Le postulat au cœur de ce travail de recherche est le suivant : l'acquisition progressive du savoir-faire corporel au cours de la vie fœtale et des premiers mois de vie post-partum est sous-tendue par deux mécanismes d'apprentissage, l'exploitation de la sensibilité aux contingencessensorimotrices et la motivation intrinsèque. Ce travail de thèse explore la première partie de ce postulat, c'est-à-dire le rôle jouée par la sensibilité aux contingences sensorimotrices dans le développement du savoir-faire corporel. Afin d'investiguer cette hypothèse, ce travail de thèsese concentre dans un premier temps sur l'analyse critique des données expérimentales déjà existantes sur le sujet, à la fois en psychologie du développement et en robotique développementale. Dans un second temps, cette recherche vise à approfondir notre compréhension de l'acquisition du savoir-faire corporel à travers l'expérimentation chez le bébé âgé de moins d'un an.
... Inspired by the comparator model as a proposed mechanism for the sense of agency in developmental psychology and cognitive neuroscience, developmental robotics researchers have taken implementations of the model as a way to imbue artificial agents with a sense of agency. In an approach collapsing sense of agency, sense of body-ownership, and sense of selfhood, Pitti et al. (2009) equipped a head-neck-eyes robot with the ability to detect contingencies in sensorimotor networks using an artificial neural network that models spike timing-dependent synaptic plasticity (STDP) as observed in the central nervous system. STDP models the process of Hebbian learning and the constituent change in connection strength between pre-and postsynaptic neurons, taking into account the need for the presynaptic neuron to fire before the postsynaptic neuron to establish proper temporal dynamics corresponding to the ascribed causal connection. ...
... STDP models the process of Hebbian learning and the constituent change in connection strength between pre-and postsynaptic neurons, taking into account the need for the presynaptic neuron to fire before the postsynaptic neuron to establish proper temporal dynamics corresponding to the ascribed causal connection. The resulting neural architecture implemented by Pitti et al. (2009) represented the system's self-produced visuomotor information, making the detection of sensorimotor contingencies possible by inspecting the clusters of neurons whose connections had been strengthened by a reinforcement learning algorithm. Over time, congruent sensorimotor neural pairs are reinforced, whilst incongruent ones are weakened and eventually inhibited. ...
... However, we counter that this is how the comparator model has often been used and interpreted in certain lines of research. This becomes, e.g., evident in artificial implementations of the sense of agency when a system that merely learned to compare its sensory predictions and observations is said to have a sense of agency (Pitti et al. 2009;Brody 2016). Moreover, experiments have been set up according to this interpretation of the model: they tap into the ability to detect the congruence of sensorimotor contingencies but fail to test for the ability to make the subsequent inference that when a match is detected, the action is likely to be caused by oneself (see e.g. ...
The development of a sense of agency is indispensable for a cognitive entity (biological or artificial) to become a cognitive agent. In developmental psychology, researchers have taken inspiration from adult cognitive psychology and neuroscience literature and use the comparator model to assess the presence of a sense of agency in early infancy. Similarly, robotics researchers have taken components of the proposed mechanism in attempts to build a sense of agency into artificial systems. In this article, we identify an invalidating theoretical flaw in the reasoning underlying this conversion from adult studies to developmental science and cognitive systems research, rooted in an oversight in the conceptualization of the comparator model as currently used in experimental practice. In these experiments, the emphasis has been put solely on testing for a match between predicted and observed sensory consequences. We argue that the match by itself can exclusively generate a simple categorization or a representation of equality between predicted and observed sensory consequences, both of which are insufficient to generate the causal representations required for a sense of agency. Consequently, the comparator model, as it has been described in the context of the sense of agency and as it is commonly used in experimental designs, is insufficient to generate the sense of agency: infants and robots require more than developing the ability to match predicted and observed sensory consequences for a sense of agency. We conclude with outlining possible solutions and future directions for researchers in developmental science and artificial intelligence.
... This learning paradigm may be implemented in several ways. For instance, in [14], the authors use an implementation based on neural networks to detect contingencies in sensorimotor networks. In [15] developmental learning is based on a set of "tripartite structure[s] comprising a context, action, and result" called schema, this structure is then completed in [16] by adding a target value, and in [17] the original schemas have been extended to "improve the original learning criteria to handle POMDP domains". ...
This paper reports preliminary research work on applying developmental learning to social robotics for making human-robot interactions more instinctive and more natural. Developmental learning is an unsupervised learning strategy relying on the fact that the learning agent is intrinsically motivated, and is able to incrementally build its own representation of the world through its experiences of interaction with it. Our claim is that using developmental learning in social robots could dramatically change the way we envision human-robot interaction, notably by giving the robot an active role in the interaction building process, and even more importantly, in the way it autonomously learns suitable behaviors over time. Developmental learning appears to be an appropriate approach to develop a form of "interactional intelligence" for social robots. In this work, our goal was to set up a common framework for implementing, experimenting and evaluating developmental learning algorithms with various social robots.
... Dans nos expériences, un robot constitué d'un bras et d'une caméra va tout d'abord effectuer un apprentissage des associations visuomotrices à l'aide de GF. Puis, une fois son schéma corporel appris, un second module de GF est ajouté pour apprendre à accéder à un nouvel "ensemble de tâches" sensorimotrices (en anglais, "task-set") [Pitti et al., 2013a[Pitti et al., , 2009, lequel doit être appris notamment suite à des transformations liées à des mouvements de la tête (rotations, translations), ou à l'utilisation d'outils. ...
Ma thèse porte sur l'intégration développementale de différents systèmes d'apprentissage dans un robot, du babillage moteur à l'émergence de l'utilisation d'outils. L'utilisation d'outils recouvre de nombreuses problématiques, certaines bas niveau (comme l'extension du schéma corporel) et d'autres plus haut niveau (comme la capacité à faire une séquence d'actions). Nous avons pour cela proposé un modèle appelé Dynamic Sensorimotor Model (DSM). DSM apprend des lois sensorimotrices, qui consistent à prédire les variations sensorielles (comme le déplacement d'un objet dans l'espace visuel) en fonction : 1) De magnitudes motrices (comme des commandes en vitesse de servomoteurs). 2) D'un contexte donné (un vecteur de données sensorielles). Un tel prédicteur peut apprendre et affiner ses lois sensorimotrices dans n'importe quelle situation, que ce soit durant l'exécution d'une tâche ou durant une phase de babillage moteur. L'apprentissage de ces prédictions est donc indépendant de l'exécution de tâches particulières, et pourra être exploité dans de nouveaux contextes, et pour satisfaire de nouvelles tâches. Pour cela, DSM contient un mécanisme de simulation motrice mais aussi un mécanisme de simulation de contextes. Ces simulations portent ainsi sur : 1) Les entrées motrices, ce qui permet de déterminer les commandes motrices à effectuer en vue d'une tâche particulière. 2) Les entrées sensorielles, ce qui permet de proposer des contextes alternatifs au sein desquels les actions permettant la réalisation d'une tâche pourront être effectuées. Ces contextes alternatifs pourront alors se constituer en sous-buts permettant d'effectuer une séquence d'actions. Grâce à ces simulations, des expériences sur robot réel ont permis de satisfaire une tâche consistant à rejoindre une cible avec l'extrémité du bras, en faisant un détour pour saisir un outil. La saisie a comme propriété d'étendre le schéma corporel (le segment terminal du bras du robot). La capacité à faire des séquences à la volée repose sur les contextes qui auront été appris. Cela met en évidence l'importance d'avoir des contextes ne contenant que les données suffisantes à la prédiction, afin de générer, par le mécanisme de simulation, des sous-buts les plus minimaux possibles pour satisfaire un but donné. Notre modèle catégorise des lois additives afin de ne pas perturber les lois sensorimotrices précédemment apprises et ainsi apprendre des lois de manière incrémentale. Dans DSM, une nouvelle catégorie se caractérise par l'instauration d'une distance entre la configuration sensorielle correspondant au contexte actuel, dans lequel les lois courantes sont en échec, et le dernier contexte dans lequel ces lois s'appliquaient correctement. Cette distance entre contextes est donc multimodale, et indépendante de la topologie propre des senseurs d'entrée. Par contre, étant issue de deux situations à deux moments différents, cette distance dépend de l'exploration sensorimotrice du robot durant cet interval de temps. Pendant cette période, les senseurs qui auront suffisamment changés de valeurs apparaîtront comme discriminant un contexte par rapport à l'autre, bien qu'ils ne soient pas tous pertinents. Ce sera par l'action que les senseurs pertinents seront sélectionnés.
