ArticlePDF Available

Imitation: a means to enhance learning of a synthetic proto-language in an autonomous robot

Authors:
Aude Billard at École Polytechnique Fédérale de Lausanne
Aude Billard
Kerstin Dautenhahn at University of Waterloo
Kerstin Dautenhahn
Chrystopher L. Nehaniv at University of Waterloo
Chrystopher L. Nehaniv
Download full-text PDF
Read full-text
Download citation

Abstract and Figures

This paper addresses the role of imitation as a means to enhance the learning of communication skills in autonomous robots. A series of robotic experiments are presented in which autonomous mobile robots are taught a synthetic proto-language. Learning of the language occurs through an imitative scenario where the robot replicates the teacher's movements. Imitation is here an implicit attentional mechanism which allows the robot imitator to share a similar set of proprio- and exteroceptions with the teacher. The robot grounds its understanding of the teacher's words, which describe the teacher's current observations, upon its own perceptions which are similar to those of the teacher. Learning of the robot is based on a dynamical recurrent associative memory architecture (DRAMA). Learning is unsupervised and results from the self-organization of the robot's connectionist architecture. Results show that the imitative behavior greatly improves the efficiency and speed of the learning. More...
Figures - uploaded by Chrystopher L. Nehaniv
Author content
All figure content in this area was uploaded by Chrystopher L. Nehaniv
Content may be subject to copyright.
Content uploaded by Chrystopher L. Nehaniv
Author content
All content in this area was uploaded by Chrystopher L. Nehaniv on Aug 15, 2015
Content may be subject to copyright.
0 100 200 300 400 500 600 700 800 900 1000
0
2
4
6
8
10
12
14
radio
compass:
180−135
compass:
135−90
IR
Light
M−left
M−right
Number of cycles
Number of teachings
Object1
Object2
Hill 1
Hill2
Hill3
... 16 Such training could be particularly relevant for robots. Billard (2002), for example, could likely train "touch your nose" with her imitation program, but would have difficulty training the difference between that command and "touch my nose" without adding some kind of inversion of meaning into the system. ...
... Nevertheless, our point is not that M/R training can necessarily engender such traits in robots or atavars, but rather that the creation of such traits in computational mechanisms (e.g. Billard, 2002;Breazeal and Scassellati, 2002) should not be seen as an end, but rather a beginning that can enable the implementation of M/R training. Such a scenario suggests how understanding the bases for imitation in animals and humans might assist in developing appropriate control programs for mechanical systems. ...
Training behavior by imitation: From parrots to people … to robots?
Article
  • Mar 2007
Mechanisms of imitation and social matching play a fundamental role in development, communication, interaction, learning and culture. Their investigation in different agents (animals, humans and robots) has significantly influenced our understanding of the nature and origins of social intelligence. Whilst such issues have traditionally been studied in areas such as psychology, biology and ethnology, it has become increasingly recognised that a 'constructive approach' towards imitation and social learning via the synthesis of artificial agents can provide important insights into mechanisms and create artefacts that can be instructed and taught by imitation, demonstration, and social interaction rather than by explicit programming. This book studies increasingly sophisticated models and mechanisms of social matching behaviour and marks an important step towards the development of an interdisciplinary research field, consolidating and providing a valuable reference for the increasing number of researchers in the field of imitation and social learning in robots, humans and animals.
View
... It is only a dozen of word-object associations that AIBO was reported to have learned in 2000 (Kaplan, 2000). 24 Research on unsupervised learning through imitation has recently incorporated limited word-action associations too (Billard, 2002). 25 Sometimes the visual data is manually segmented into single-objects, cf. for example (Cottrell et al., 1990), or it is particularly chosen/acquired so that it depicts a single object. ...
Vision -Language Integration: a Double-Grounding Case
Thesis
Full-text available
  • Jan 2005
This thesis explores the issue of vision-language integration from the Artificial Intelligence perspective of building intentional artificial agents able to combine their visual and linguistic abilities automatically. While such a computational vision-language integration is a sine qua non requirement for developing a wide range of intelligent multimedia systems, the deeper issue still remains in the research background. What does integration actually mean? Why is it needed in Artificial Intelligence systems, how is it currently achieved and how far can we go in developing fully automatic vision-language integration prototypes? Through a parallel theoretical investigation of visual and linguistic representational systems, the nature and characteristics of the subjects of this integration study, vision and language, are determined. Then, the notion of their computational integration itself is explored. An extensive review of the integration resources and mechanisms used in a wide-range of vision-language integration prototypes leads to a descriptive definition of this integration as a process of establishing associations between images and language. The review points to the fact that state of the art prototypes fail to perform real integration, because they rely on human intervention at key integration stages, in order to overcome difficulties related to features vision and language inherently lack. In looking into these features so as to discover the real need for integrating vision and language in multimodal situations, intentionality-related issues appear to play a central role in justifying integration. These features are correlated with Searle's theory of intentionality and the Symbol Grounding problem. This leads to a view of the traditionally advocated grounding of language in visual perceptions as a bi-directional, not one-directional, process. It is argued that vision-language integration is rather a case of double-grounding, in which linguistic representations are grounded in visual ones for getting direct access to the physical world, while visual representations, in their turn, are grounded in linguistic ones for acquiring a controlled access to mental aspects of the world. Last, the feasibility of developing a prototype able to achieve this double-grounding with minimal human intervention is explored. VLEMA is presented, a prototype which is fed with automatically reconstructed building-interior scenes, which it subsequently describes in natural language. The prototype includes a number of unique features which point to new directions in building agents endowed with real vision-language integration abilities.
View
... While the system uses a recurrent neural network -which ''encodes" sensorimotor trajectories for later recall -to learn behaviors, it can also be interpreted in terms of a mirror neuron system in that the encoding and imitation of the action are analogous to observation and motor generation. Additionally, robot imitation learning has drawn from the concept of mirror neurons in the past, imitating these systems through hidden Markov models (Amit & Mataric, 2002) or neural networks (Billard, 2002). In an interactive context, such a mechanism could enable a robot to synchronize its movements with a human team member, adapting to dynamic situations and generating learned behaviors as required. ...
View
... In the case of the virtual butler, learning mechanisms need to be based on episodic/case-based (rather than statistical) [11,26] events in order to facilitate one-shot learning. Such an approach would naturally lead to the possibility of the immediate re-use (imitation) of user's behaviours by the robot [5,12,37] in just the same way that people mirror each other/align in body posture and conversation. ...
Spoken Language Processing: Where Do We Go from Here?
Chapter
  • Jan 2013
Recent years have seen steady improvements in the quality and performance of speech-based human-machine interaction driven by a significant convergence in the methods and techniques employed. However, the quantity of training data required to improve state-of-the-art systems seems to be growing exponentially, and yet performance appears to be reaching an asymptote that is not only well short of human performance, but which may also be inadequate for many real-world applications. This situation suggests that there may be a fundamental flaw in the underlying architecture of contemporary speech-based systems, and the future direction for research into spoken language processing is currently uncertain. This chapter addresses these issues by stepping outside the familiar domains of speech science and technology, and instead draws inspiration from recent findings in fields of research that are concerned with the neurobiology of living systems in general. In particular, four areas are highlighted: the growing evidence for an intimate relationship between sensor and motor behaviour in living organisms, the power of negative feedback control to accommodate unpredictable disturbances in real-world environments, mechanisms for imitation and mental imagery for learning and modelling, and hierarchical models of temporal memory for predicting future behaviour and anticipating the outcome of events. The chapter shows how these results point towards a novel architecture for speech-based human-machine interaction that blurs the distinction between the core components of a traditional spoken language dialogue system; an architecture in which cooperative and communicative behaviour emerges as a by-product of a model of interaction where the system has in mind the needs and intentions of a user, and a user has in mind the needs and intentions of the system. It concludes with a roadmap of technical pre-requisites and desiderata that would seem to be necessary if voice-based interaction with an autonomous agent such as a virtual butler is to become a practical reality.
View
... On pourrait dire que ce lien entre couche conceptuelle et sensori-motrice, au coeur des sciences cognitives actuelles, est aussi le point sensible de la robotique, exacerbé en robotique de la parole par les propriétés particulièrement complexes du langage humain. Cette problématique est aussi celle du Laboratoire d'Algorithmes et Systèmes d'Apprentissage de l'Ecole Polytechnique Fédérale de Lausanne et notamment des travaux d'Aude Billard sur l'apprentissage gestuel par imitation chez les robots (Calinon et Billard, 2007;Billard, 2002). ...
From substance to form: the role of speech orofacial and brachiomanual constraints in the emergence of language
Article
Full-text available
  • Oct 2007
And what if the sensori-motor properties of speech model language ? This hypothesis drove language in the field of complexity and embodied cognition. Here, we introduce different kind of evidences showing the part of the motricity of speech in the genesis of language. Orofacial motricity, first, with the assumption that the properties of inter-articulators coordination may constraint the morphogenesis of langage. Orofacial and brachiomanual motricity, then, with the hypothesis that language may emerge from hand-mouth coordination that support the act of pointing by the voice and by the hand. Hence, our experiments analyze the recorded motions of speakers of french during differents tasks in order to establish the properties of jaw-tongue-lips coordinations in speech and of jaw-hand coordination in pointing. These studies lie whithin the global and recent research framework that propose to investigate language as a complex-system.
View
... They presented a set of language games in which two agents agree on the semantic and syntax of the vocabulary that identify the objects in their shared environment. Billard [6] claimed that imitation can be used to enhance autonomous robots' learning of communication skills. The sharing of a similar perceptual context between the imitator and the demonstrator can create the necessary social context where language can develop in. ...
On the Evolution of Behaviors through Embodied Imitation
Article
Full-text available
  • May 2015
This article describes research in which embodied imitation and behavioral adaptation are investigated in collective robotics. We model social learning in artificial agents with real robots. The robots are able to observe and learn each others' movement patterns using their on-board sensors only, so that imitation is embodied. We show that the variations that arise from embodiment allow certain behaviors that are better adapted to the process of imitation to emerge and evolve during multiple cycles of imitation. As these behaviors are more robust to uncertainties in the real robots' sensors and actuators, they can be learned by other members of the collective with higher fidelity. Three different types of learned-behavior memory have been experimentally tested to investigate the effect of memory capacity on the evolution of movement patterns, and results show that as the movement patterns evolve through multiple cycles of imitation, selection, and variation, the robots are able to, in a sense, agree on the structure of the behaviors that are imitated.
View
... The relation between visual representations and recognition of bodily action on the one hand and the bodily action itself has already been intensely studied in robotics research, particularly in research on imitation (Billard 2002;Demiris & Johnson 2003). It is moreover a key topic in 'embodied artificial intelligence' (Pfeifer & Bongard 2006) which emphasises the grounding of cognition in bodily activities. ...
Emergent Mirror Systems for Body Language
Chapter
Full-text available
  • Jan 2012
This chapter investigates how a vocabulary for talking about body actions can emerge in a population of grounded autonomous agents instantiated as humanoid robots. The agents play a Posture Game in which the speaker asks the hearer to take on a certain posture. The speaker either signals success if the hearer indeed performs an action to achieve the posture or he shows the posture himself so that the hearer can acquire the name. The challenge of emergent body language raises not only fundamental issues in how a perceptually grounded lexicon can arise in a population of autonomous agents but also more general questions of human cognition, in particular how agents can develop a body model and a mirror system so that they can recognize actions of others as being the same as their own.
View
... (Goldberg & Mataric 1999) has developed augmented markov models as a practical tool for modeling interaction dynamics between a robot and its environment; this form of model is well suited to the process modeling needed in role transfer. (Billard 2001) has demonstrated some simple mimicry and sequence learning for a robot doll and an instrumented human. The DRAMA architecture she uses is well suited to situations where the robot's state can be represented by a simple, clean vector of binary percepts. ...
SOLUTION APPROACH PROBLEM RELATED IN THE FIELD OF SERVICE ROBOT TASK PLANNING
Conference Paper
Full-text available
  • Jan 2005
This paper presents solution approach task planning issues in field of service mobile robot should be performed efficiently, due to real time requirement of service robot interact with various environment. The robotic system performance depends upon the user, task (Voice), speech recognition, natural language processing, intelligence and size of world states. Thus, in service robotic applications, where the above components can be large, planning may turn inefficient and even unsolvable. Because unfortunately there are different drawbacks within the service robotic system such as no possibility of real time reactions to dynamic environmental changes, combinational explosion during the expansion search space and complexity of the environmental model. In the artificial intelligence literature on planning, attention has been put into efficient management of large scale world descriptions. In the large scale situations, artificial intelligence planners may consume intractable amount of storage and computing time, due to the huge amount of information. So the task planning system must be able to generate a sequence of elementary commands (Ordinary meaning of executable language), in order to transform the current situation into the goal situation.
View
... Robotics research has long been inspired by human social learning, and this work has largely focused on one aspect-imitation. This has been called Learning by Imitation or Demonstration and works toward robots capable of reproducing demonstrated motor actions [29], learning generalized task representations [26], policies [9] or a proto-language about actions [3]. ...
View
How can a recurrent neurodynamic predictive coding model cope with fluctuation in temporal patterns? Robotic experiments on imitative interaction
Article
  • Mar 2017
The current paper examines how a recurrent neural network (RNN) model using a dynamic predictive coding scheme can cope with fluctuations in temporal patterns through generalization in learning. The conjecture driving this present inquiry is that a RNN model with multiple timescales (MTRNN) learns by extracting patterns of change from observed temporal patterns, developing an internal dynamic structure such that variance in initial internal states account for modulations in corresponding observed patterns. We trained a MTRNN with low-dimensional temporal patterns, and assessed performance on an imitation task employing these patterns. Analysis reveals that imitating fluctuated patterns consists in inferring optimal internal states by error regression. The model was then tested through humanoid robotic experiments requiring imitative interaction with human subjects. Results show that spontaneous and lively interaction can be achieved as the model successfully copes with fluctuations naturally occurring in human movement patterns.
View
View
Recent Advances in Drug Development
Chapter
  • Jan 2015
Encouraging findings are being reported in the development of drugs for the treatment of addictive disorders. This chapter reviews drug development for the treatment of alcohol, cannabis, cocaine, opiates, and methamphetamine dependence. Gabapentin, pregabalin, ondansetron, and sertraline have been investigated for the management of alcohol dependence in double-blind, placebo-controlled designs. Gabapentin has been shown in two studies to reduce the emergence and percentage of heavy drinking days following alcohol detoxification treatment. Pregabalin was shown to reduce withdrawal symptomatology, craving, and relapse. Sertraline was reported to be efficacious in reducing drinking days and heavy drinking days in study participants with late-onset alcoholism. Ondansetron was shown to reduce drinks per drinking day and to increase the percentage of days abstinent in an alcohol-dependent population. Gabapentin has been shown to reduce cannabis use, withdrawal, and craving relative to a placebo group in one study. Nabilone has been shown to reduce marijuana withdrawal symptoms and self-administration in a clinical laboratory study, providing a rationale for an outpatient trial. A combination of mixed amphetamine salts and topiramate, compared to a placebo group, produced a greater percentage of cocaine-dependent participants achieving 3 weeks of continuous abstinence. Bupropion, methylphenidate, naltrexone, and topiramate have been evaluated for efficacy in facilitating abstinence in methamphetamine dependence. Bupropion has given the strongest signals of efficacy. Topiramate may have some utility as a relapse prevention agent but this needs to be confirmed. Implantable and injectable forms of buprenorphine are being developed for the treatment of opioid dependence.
View
What infants' imitations communicate: With mothers, with fathers and with peers.
Chapter
  • Jan 1999
Reviews the literature on the following aspects of imitation: (1) Imitative reactions, sympathetic emotions and human learning; (2) What is imitated?; (3) Constitutional foundations for imitation of motives; (4) Antenatal developments; (5) Evolution of imitation by motive reflection of mirror affordances; (6) How psychological theory has failed to explain human mimesis; (7) Imitation as communication; (8) The neonate's inborn plan for matching with the bodies and actions of persons: How they behave when they imitate; (9) Imitation cycles microanalysed in communication with infants; (10) Protoconversational interaction with imitation; (11) The state of the imitative mechanism before term; (12) Developments in infant's motives to communicate, and changing imitations; (13) Imitation in different relationships, I: Fathers and mothers with infant boys and girls; (14)Varieties of companionship of infants with fathers and mothers in Cretan families; (15) Imitation in different relationships, II: Infants imitating infants; and (16) Imitating in different relationships, III: Infants and mirrors. It is concluded that infants are born ready to reciprocate in rhythmic engagements with the motives of sympathetic partners, and that imitations are made communicatively. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
View
View
Imitation: A Review and Critique
Article
  • Jan 1973
Imitation in nonprimates is reviewed along theoretical and experimental lines. Difficulties with definitions and classifications of imitative phenomena are noted. The history of the topic is considered in terms of three basic paradigms: (1) early comparative psychology, (2) S-R/reinforcement learning theory, and (3) classical ethology. The current status of work on bird vocalization (“vocal imitation” and “vocal mimicry”), “social facilitation,” and “observational learning” is evaluated. It is concluded that an inductive approach is needed in the study of imitative behavior, starting with the selection of reliably observed, naturally occurring imitative phenomena, followed by appropriate experimental analyses of the various determinants of each phenomenon.
View
View
View
Child's Talk: Learning to Use Language
Article
  • Jan 1985
To carry out his investigations, Bruner went to "the clutter of life at home," the child's own setting for learning, rather than observing children in a "contrived video laboratory." For Bruner, language is learned by using it. An central to its use are what he calls "formats," scriptlike interactions between mother and child in short, play and games. What goes on in games as rudimentary as peekaboo or hide-and-seek can tell us much about language acquisition.But what aids the aspirant speaker in his attempt to use language? To answer this, the author postulates the existence of a Language Acquisition Support System that frames the interactions between adult and child in such a way as to allow the child to proceed from learning how to refer to objects to learning to make a request of another human being. And, according to Bruner, the Language Acquisition Support System not only helps the child learn "how to say it" but also helps him to learn "what is canonical, obligatory, and valued among those to whom he says it." In short, it is a vehicle for the transmission of our culture."
View
Avian Movement Imitation and a New Form of Mimicry: Tracing the Evolution of a Complex Form of Learning
Article
  • Sep 1992
  • BEHAVIOUR
1. The spontaneous imitation of movements, previously known in two orders of mammals, is demonstrated in a psittacine bird. 2. The animal, a Grey parrot which has bonded to humans, utilizes its torso, legs, wings, head, beak, and tongue in the imitation of human movements. 3. A new form of imitation, ''non-vocal mimicry,'' is tentatively identified. It is defined by the use of skeletal movements to produce mimetic sounds. 4. Hierarchical relationships and phylogenetic patterns of occurrence suggest that imitative learning in birds may have evolved through the sequence: song/call learning --> vocal mimicry --> non-vocal mimicry --> movement imitation. 5. These relationships and patterns, and possible differences in function and incubation time, suggest that movement imitation in birds is not homologous to that in mammals.
View
The symbol grounding problem
Article
  • Jan 1990
  • PHYSICA D
View