Content uploaded by Silvia Coradeschi
Author content
All content in this area was uploaded by Silvia Coradeschi
Content may be subject to copyright.
Noname manuscript No.
(will be inserted by the editor)
A Short review of Symbol Grounding in Robotic and
Intelligent Systems
Silvia Coradeschi · Amy Loutfi · Britta Wedre
Received: date / Accepted: date
Abstract This paper gives an overview of the research
papers published in Symbol Grounding in the period
from the beginning of the 21st century up 2012. The
focus is in the use of symbol grounding for robotics
and intelligent system. The review covers a number of
subtopics, that include, physical symbol grounding, so-
cial symbol grounding, symbol grounding for vision sys-
tems, anchoring in robotic systems, and learning sym-
bol grounding in software systems and robotics. This
review is published in conjunction with a special is-
sue on Symbol Grounding in the K¨unstliche Intelligenz
Journal.
Keywords Symbol Grounding · Anchoring · Cognitive
Robotics · Social Symbol Gronding
1 Introduction
The main dream of Artificial Intelligence has been to
create autonomous and intelligent systems that can rea-
son and act in the real world. For such a dream to
become true an essential ingredient is to establish and
maintain a connection between what the system reasons
about and what it can sense in the real world. This can
be considered as an aspect of the Symbol Grounding
Problem. The Symbol Grounding Problem (SGP) has
been defined by Harnad in [36] as the problem of how
to ground the meanings of symbol tokens in anything
different than other (meaningless) symbols. Since its
definition, symbol grounding has been an area of inter-
est both in the fields of psychology as well as artificial
¨
Orebro University, AASS
Tel.: +46-19-303000
E-mail: silvia.coradechi@oru.se E-mail: amy.loutfi@oru.se
intelligence. Its practical application has also been stud-
ied in robotics and intelligent systems, with particular
emphasis on the problem of grounding symbols to the
data acquired by physically embedded sensors. It is this
practical application which is the focus of this paper.
The review covers the recent literature in the subject
and in particular the period from 2000 to 2012 and is
organized into two subtopics which relate to the cur-
rent approaches to SGP in robotics and intelligent sys-
tems: Physical Symbol Grounding and Social Symbol
Grounding. The ”Physical Symbol Grounding” as been
defined by Vogt in [101] as the grounding of symbols to
real world objects by a physical agent interacting in the
real world; while its social component, ”Social Symbol
Grounding”, refers to the collective negotiation for the
selection of shared symbols (words) and their grounded
meanings in (potentially large) populations of agents as
defined by Cangelosi in [12].
These are both significant and hard problems. As
explained in [101] Physical Symbol Grounding requires
constructing a consistent relation between percepts that
may vary under different conditions, and which often
have a high dimensionality. Categorising the dimen-
sionalities may yield different categories, which how-
ever should be relate to one concept often with the help
of invariant feature detectors. According to Vogt [104]
the social symbol grounding problem may even be an
harder problem to solve, because to learn what a word-
form refers to can result in Quine’s referential indeter-
minacy problem: the unknown word can -theoretically-
refer to an infinite number of objects. Vogt investigated
in [104] a number of heuristics from child language ac-
quisition literature that help to reduce this indetermi-
nacy: joint attention, principle of contrast and correc-
tive feedback. In [103] mutual exclusivity, and a few
2 Silvia Coradeschi et al.
potential dialogues that help to reduce referential inde-
terminacy have been also implemented.
It is worth to note that a recent review of Symbol
Grounding has been published in 2005 by Taddeo [95]
which specifically addresses SGP as a general problem
from a philosophical perspective. A volume edited by
Belpaeme [4] presents current views on symbol ground-
ing both from a philosophical and robotics perspec-
tive.This review is complementary as it focuses on work
of more practical relevance to robotics and intelligent
systems. Finally Cangelosi in [14] discusses the current
progress and solutions to the symbol grounding prob-
lem and specifically identifies which aspects of the prob-
lem have been addressed and issues and scientific chal-
lenges that still require investigation.
It is the authors belief that the focus of this review
is especially timely as the steps towards the solution of
the SGP will be key to creating the next generation of
robotic systems that are capable of high level reasoning.
The review is structured in a number of subtopics.
In the Physical Symbol Grounding section learning of
categories on the basis of sensor data and grounding
of actions are considered. In addition the concept of
Anchoring of symbols to sensor data is defined and the
work in this topic is summarized. The review ends with
a summary of works in Social Symbol Grounding and
works on Symbol Grounding applied the the semantic
web.
2 Physical Symbol Grounding
When dealing with the Physical Symbol Grounding,
one of the basic challenges examined in the literature is
to ground symbols to perceptual representations (sen-
sor data), where the symbols denote categorical con-
cepts such as color, shape and spatial features. Typi-
cally, the sensor data come from vision sensors but other
modalities have also been used. The methods explored
are often inspired by connectionist models and a wide
range of learning algorithms have been applied. Unsu-
pervised methods have been investigated by Vavrecka
in [?] where a biologically inspired model for ground-
ing spatial terms is presented. Color, shape and spa-
tial relations of two objects in 2D space are grounded.
Images with two objects are presented to an artificial
retina and five-word sentences describing them (e.g.
”Red box above green circle”) are inputed. The im-
plementation is done using Self-Organizing Map and
Neural Gas algorithms. The Neural Gas algorithm is
found to lead to better performance especially in case
of scenes with higher complexity. In [50] Kittler con-
siders a visual bootstrapping approach for the unsu-
pervised symbol grounding is presented. The method is
based on a recursive clustering of a perceptual category
domain controlled by goal acquisition from the visual
environment.
A supervised method is used in a framework for
modeling language in neural networks and adaptive agent
simulations by Cangelosi [11]. In this work symbols are
directly grounded into the agents’ own categorical rep-
resentations and have syntactic relationships with other
symbols. The grounding of basic words, acquired via di-
rect sensorimotor experience, is transferred to higher-
order words via linguistic descriptions.
A few works consider the combination of both vi-
sual and auditory data, where the combination gives a
better result than using one modality alone. In [105] a
multimodal learning system is presented by Yu that
can ground spoken names of objects in their physi-
cal referents and learn to recognize those objects si-
multaneously from vocal and vision input. The system
collects image sequences and speech input while users
perform natural tasks and grounds spoken names of
objects in visual perception, also learning to catego-
rize visual objects using teaching signals encoded in co-
occurring speech. Also Nakamura uses in [70, 71] vision
and speech for multimodal categorization and words
grounding by robots. The robot uses its physical em-
bodiment to grasp and observe an object from various
view points, as well as to listen to the sound during the
observing period. The method used is Latent Dirich-
let allocation (LDA)-based framework and experimen-
tal results with 40 objects (eight categories) show an
improvement with respect to just visual categorization
and show the possibility of a conversation between a
user and the robot based on the grounded words. In [72]
a system involving vision and audio data is presented
by Needham that is capable of autonomously learning
concepts (utterances and object properties) from per-
ceptual observations of dynamic scenes. This work goes
beyond categorical learning and learns also protocols
from the perceptual observations. The motivation is the
development of a synthetic agent that can observe a
scene containing interactions between unknown objects
and agents, and learn models of these sufficient to act
in accordance with the implicit protocols present in the
scene. The system is tested by learning the protocols
of simple table-top games where perceptual classes and
rules of behaviors from real world audio-visual data is
learnt in an autonomous manner.
Additional sensor modalities have been used by Groll-
man in [35] where symbol grounding in robot perception
is considered through a data-driven approach deriving
categories from robot sensor data that include infrared,
sonar and data from a time-of-flight distance camera.
Isomap nonlinear dimension reduction and Bayesian clus-
A Short review of Symbol Grounding in Robotic and Intelligent Systems 3
tering (Gaussian mixture models) with model identifi-
cation techniques are used to discover categories. Trials
in various indoor and outdoor environments with dif-
ferent sensor modalities are presented and the learned
categories are then used to classify new sensor data.
2.1 Perceptual Anchoring
A special case of Symbol Grounding is the connection
of sensor data coming from physical objects to higher
level symbolic information that refers to those objects.
The process of creating and maintaining this connection
is called Anchoring and has been formally defined by
Coradeschi in [21] and then in [22].
The use of anchoring in planning, recovery plan-
ning and solving of ambibuities is explored in works of
Karlsson and Broxvall [7, 8, 49] Anchoring with other
sensor modalities like olfaction is explored in works
of Loutfi and Broxvall [9, 59–61] while the integration
of high-level conceptual knowledge on a single agent,
via the combination of a fully-fledged Knowledge Rep-
resentation and Reasoning (KR&R) system with the
anchoring framework and more specifically, the use of
semantic knowledge and common-sense information so
as to enable reasoning about the perceived objects at
the conceptual level has been considered by Lemaignan
and Daoutis in [26, 55]. Cooperative anchoring among
robots in a robot soccer application is presented by
LeBlanc in [54] while multi-agent anchoring in a smart
home environment is presented in works of Broxvall and
Daoutis [9, 24].
A framework for computing the spatial relations be-
tween anchors is presented by Melchert in [62, 65, 66]
where a set of binary spatial relations were used to
provide object descriptions. Human interaction is used
to disambiguate between visually similar objects. Sim-
ilarly in [67] an approach to establish joint object ref-
erence is formulated by Moratz. The object recogni-
tion approach assigns natural categories (e.g. ”desk”,
”chair”, ”table”) to new objects based on their func-
tional design, relations (e.g. ”the briefcase to the left
of the chair”) are then established allowing users to re-
fer to objects which cannot be classified reliably by the
recognition system alone.
Anchoring has also been used by Lemaignan [56]
to enable a grounded and shared model of the world
that is suitable for dialogue understanding. Realistic
human-robot interactions are considered that deal with
complex, partially unknown human environments and a
fully embodied (with arms, head,...) autonomous robot
that manipulates a large range of household objects. A
knowledge base models the beliefs of the robot and also
every other cognitive agent the robot interacts with. A
framework is also presented to extract symbolic facts
from complex real scenes. The robot builds a 3D model
of the world on-line by merging different sensor modali-
ties. It computes spatial relations between perceived ob-
jects in realtime and the system allows virtually viewing
of the same scene from different points of view.
A different approach to anchoring is presented by
Heintz in [39,41,42] where anchoring is considered in the
context of unmanned aerial vehicles. In their stream-
based hierarchical anchoring framework, a classification
hierarchy is associated with expressive conditions for
hypothesizing the type and identity of an object given
streams of temporally tagged sensor data. A metric
spatio-temporal logic is used to represent the conditions
which are efficiently evaluated over these streams using
a progression-based technique. The anchoring process
constructs and maintains a set of object linkage struc-
tures representing the best possible hypotheses at any
time. Each hypothesis can be incrementally generalized
or narrowed down as new sensor data arrives. Symbols
can be associated with an object at any level of classifi-
cation, permitting symbolic reasoning on different levels
of abstraction.
Additional approaches of anchoring are presented in
a special issue on Anchoring published by the Robotics
and Autonomous Systems Journal. In [89] an overview
of the GLAIR approach to anchoring is outlined by
Shapiro where abstract symbolic terms that denote an
agent’s mental entities are anchored to the lower-level
structures used by the embodied agent to operate in
the real (or simulated) world. In [102] the anchoring
problem is approached by Vogt using semiotic symbols
defined by a triadic relation between forms, meanings
and referents. Anchors are formed between these three
elements and a robotic experiment based on adaptive
language games is presented that illustrates how the
anchoring of semiotic symbols can be achieved in a
bottom-up fashion. Person tracking using anchoring has
been investigated by Fritsch in [30] where laser range
data is used to extract the legs of a person while cam-
era images from the upper body part are used for ex-
tracting the faces. The results of the different percepts,
which originate from the same person are combined in
one anchor for the person.
An interesting application of Anchoring is in the
field of topological maps and in particular the inves-
tigation of the connection of symbolic information to
spatial information. Work in this area has been pre-
sented by Galindo in [31,32] where a multi-hierarchical
approach is used to acquire semantic information from
a mobile robot sensors for navigation tasks. The spa-
tial information is anchored to the semantic information
and the approach is validated via experiments where a
4 Silvia Coradeschi et al.
mobile robot uses and infers new semantic information
from its environment, improving its operation. Simi-
larly Elmogy in [27] investigates how a topological map
is generated to describe relationships among features of
the environment in a more abstract form to be used in
a robot navigation system. A language for instructing
the robot to execute a route in an indoor environment is
presented where an instruction interpreter processes a
route description and generates its equivalent symbolic
and topological map representations. Finally Blodow
in [6] uses semantic mapping in kitchen environments
to help performing manipulation tasks.
3 Grounding Words in Action
The research group headed by Cangelosi has been work-
ing in cognitive robotics models using the humanoid
robot iCub. In [92,93] a cognitive robotics model is de-
scribed in which the linguistic input provided by the
experimenter guides the autonomous organization of
the knowledge of the iCub. A hierarchical organiza-
tion of concepts is used for the acquisition of abstract
words. Higher-order concepts are grounded using ba-
sic concepts and actions that are directly grounded in
sensorimotor experiences. The method used is a re-
current neural network that permits the learning of
higher-order concepts based on temporal sequences of
action primitives. In [13] a review of cognitive agent
and developmental robotics models of the grounding of
language is presented. Three models are discussed: a
multi-agent simulation of language evolution, a simu-
lated robotic agent model for symbol grounding trans-
fer, and a model of language comprehension in the hu-
manoid robot iCub. The complexity of the agent’s sen-
sorimotor and cognitive system gradually increases in
the three models. In previous works [15, 17] the combi-
nation of cognitive robotics with neural modeling method-
ologies is also considered to demonstrate how the lan-
guage acquired by robotic agents can be directly grounded
in action representations, in particular language learn-
ing simulations show that robots are able to acquire
new action concepts via linguistic instructions. Finally
in [16] an embodied model for the grounding of language
in action is presented and experimented on epigenetic
robots. Epigenetic robots have an integrative vision of
language in which linguistic abilities are strictly depen-
dent on and grounded in other behaviors and skills. Ex-
periments done with simulated robots show that higher
order behavioral abilities can be autonomously built on
previously grounded basic action categories following
linguistic interaction with human users.
Another approach to learning of actions is presented
by Oladell in [75] where representational complexity is
managed using a symbolic feature representation gener-
ated via policies, affordances and goals. The approach
is demonstrated in a simulation environment with a
robot arm and camera. Learning tasks revolve around
lift, move, and drop and the policies are learnt using
QLearning. The agent learns new policies, affordances
and goals and adds them to the dictionary. After each
addition, the best common sub-structure is extracted.
Learning of meanings of both action and substantive
words is presented by Tellex in [97] where a probabilis-
tic approach is used to learn word meanings from large
corpora of examples and use those meanings to find
good groundings in the external world. The framework
handles complex linguistic structures such as referring
expressions (for example, ”the door across from the el-
evators”) and multiargument verbs (for example, ”put
the pallet on the truck”) by dynamically instantiating
a conditional probabilistic graphical model that factors
according to the compositional and hierarchical struc-
ture of a natural language phrase.
4 Social Symbol Grounding
A recent line of research in Symbol Grounding is So-
cial Symbol Grounding. As defined by Cangelosi [12]
the social symbol grounding considers the next step af-
ter the connections between the sensor data and sym-
bols for individual agents are achieved, that is how can
these connections be shared among many agents. Sev-
eral approaches have been presented to address this is-
sue. Heintz in [40] presents a distributed information
fusion system for collaborative UAVs. In [90] Steels ex-
amines if a perceptually grounded categorical repertoire
can become sufficiently shared among the members of
a population to allow successful communication, using
color categorization as a case study. Several models are
proposed that are inspired by alternative hypotheses of
human categorization. He has proposed various robotic
models of the emergence of communication based on
the languages games for the Talking Heads experiments
and the AIBO and QRIO robots.The paper argues that
the collective choice of a shared repertoire must inte-
grate multiple constraints, including constraints com-
ing from communication. Similarly Fontanari in [29] use
language games to study evolution of compositional lex-
icons. In [104] the New Ties project is presented. The
project aims at evolving a virtual simulated cultural
society where the agents evolve a communication sys-
tem that is grounded in their interactions with their
virtual environment and with other individuals. An hy-
brid model of language learning involving joint atten-
tion, feedback, cross-situational learning and the prin-
ciple of contrast is investigated. A number of experi-
A Short review of Symbol Grounding in Robotic and Intelligent Systems 5
ments are carried out in simulation showing that levels
of communicative accuracy better than chance evolve
quite rapidly and that accuracy is mainly achieved by
the joint attention and cross-situational learning mech-
anisms while feedback and the principle of contrast con-
tribute less. As mentioned in the introduction the so-
cial symbol grounding problem is a difficult problem to
solve, because an unknown word can -theoretically- re-
fer to an infinite number of objects. Vogt investigated
in [104] a number of heuristics from child language ac-
quisition literature that help to reduce this indetermi-
nacy: joint attention, principle of contrast and correc-
tive feedback. In [103] mutual exclusivity, and a few
potential dialogues that help to reduce referential in-
determinacy have been also implemented. In [94] it is
argued that the primary motivation for an agent to con-
struct a symbol-meaning mapping is to solve a tasks,
in particular it is investigated how agents learn to solve
multiple tasks and extract cumulative knowledge that
helps them to solve each new task more quickly and
accurately.
5 Grounding symbols in the Semantic Web
Recent trends examine the Symbol Grounding Prob-
lem in the context of web technologies and specifically
the semantic web. In [46] Semantic Web technologies
are used by Johnston for grounding robotic systems. In
particular the OBOC robotic software system includ-
ing an ontology-based vision subsystem is presented.
OBOC has been tested and evaluated in the robot soc-
cer domain. The grounding of knowledge for everyday
tasks using the World Wide Web has been considered
by Nyga and Tenorth in [74,98] while a first attempt of
an extension of the anchoring framework to handle the
grounding and integrate symbolic and perceptual data
that are available on the web is outlined by Daoutis
in [25].
The problem of giving semantics to the semantic
web is considered by Cregan in [23]. The paper argues
that the symbol grounding problem is of relevance for
the Semantic Web as inappropriate correspondence be-
tween symbol and referent can result in logically valid
but meaningless inferences. In fact ontology languages
can provide a means to relate data items to each other
in logically well-defined ways, but they are intricate
”castles in the air” without a pragmatic semantics link-
ing them in a systematic and unambiguous way to the
real world entities they represent.
6 Conclusions
This short review presents recent work in Symbol Ground-
ing that is focused on the use of symbol grounding in
robotics and intelligent systems applications. The field
is clearly very active and many articles have been pub-
lished in recent years. This is a consequence of the cur-
rent trends of integrating robots and distributed sys-
tems in unstructured and dynamic environments. Such
environments require a flexible handling of knowledge
and the connection of symbolic and sensory informa-
tion to be able to successfully operate. In addition sys-
tems where humans have an active role are becoming
more common. Here symbol grounding is essential to
insure meaningful natural language communication. Fi-
nally the use of the web as a source of information about
objects and their properties is providing new opportuni-
ties to access a very large and updated storage of data,
both symbolic and visual. The use of symbol grounding
to connect the information in the web to real data is
maybe the most important challenge for the field.
Acknowledgements We would like to thank Tony Belpaeme,
Fredrik Heintz, Sven Albrecht, Angelo Cangelosi, Pual Vogt,
and Sverin Lemaignan for their helpful comments to improve
the article and make it more complete.
References
1.
2. S. Harnad A. Cangelosie. The adaptive advantage of
symbolic theft over sensorimotor toil: Grounding lan-
guage in perceptual categories. Evolution of Communi-
cation, 4(1):117–142, 2001.
3. T. Belpaeme and S.J. Cowley. Extending symbol
grounding. Interaction Studies, 8(1):1–6, 2007.
4. T. Belpaeme, S.J. Cowley, and K. MacDorman, editors.
Symbol grounding. John Benjamins, Amsterdam, 2009.
5. Tony Belpaeme and Stephen J. Cowley. Extending sym-
bol grounding. Interaction Studies, 8(1):1–16, 2007.
6. Nico Blodow, Lucian Cosmin Goron, Zoltan-Csaba Mar-
ton, Dejan Pangercic, Thomas R¨uhr, Moritz Tenorth,
and Michael Beetz. Autonomous semantic mapping
for robots performing everyday manipulation tasks in
kitchen environments. In 2011 IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS),
2011.
7. M. Broxvall, S. Coradeschi, L. Karlsson, and A. Saffiotti.
Have another look on failures and recovery planning in
perceptual anchoring. In AAAI Workshop - Technical
Report, volume WS-04-03, pages 63–70, 2004.
8. M. Broxvall, S. Coradeschi, L. Karlsson, and A. Saffiotti.
Recovery planning for ambiguous cases in perceptual
anchoring. In Proceedings of the National Conference
on Artificial Intelligence, volume 3, pages 1254–1260,
2005.
9. M. Broxvall, S. Coradeschi, A. Loutfi, and A. Saffiotti.
An ecological approach to odour recognition in intelli-
gent environments. In Proceedings - IEEE International
6 Silvia Coradeschi et al.
Conference on Robotics and Automation, volume 2006,
pages 2066–2071, 2006.
10. N. J. Butko and J. R. Movellan. Detecting contin-
gencies: An infomax approach. Neural Networks, 23(8-
9):973–984, 2010.
11. A. Cangelosi. Symbol grounding in connectionist and
adaptive agent models. In Lecture Notes in Computer
Science, volume 3526, pages 69–74, 2005.
12. A. Cangelosi. The grounding and sharing of symbols.
Pragmatics and Cognition, 14(2):275–285, 2006.
13. A. Cangelosi. Grounding language in action and percep-
tion: From cognitive agents to humanoid robots. Physics
of Life Reviews, 7(2):139–151, 2010.
14. A. Cangelosi. Solutions and open challenges for the sym-
bol grounding problem. International Journal of Signs
and Semiotic Systems, 1(1):49–54, 2011.
15. A. Cangelosi, E. Hourdakis, and V. Tikhanoff. Lan-
guage acquisition and symbol grounding transfer with
neural networks and cognitive robots. In IEEE Inter-
national Conference on Neural Networks - Conference
Proceedings, pages 1576–1582, 2006.
16. A. Cangelosi and T. Riga. An embodied model for senso-
rimotor grounding and grounding transfer: Experiments
with epigenetic robots. Cognitive Science, 30(4):673–
689, 2006.
17. A.a Cangelosi, V.a Tikhanoff, J.F.b Fontanari, and E.c
Hourdakis. Integrating language and cognition: A cog-
nitive robotics approach. IEEE Computational Intelli-
gence Magazine, 2(3):65–70, 2007.
18. A.a Chella, S.b Coradeschi, M.c Frixione, and A.b Saf-
fiotti. Perceptual anchoring via conceptual spaces. In
AAAI Workshop - Technical Report, volume WS-04-03,
pages 40–45, 2004.
19. H. Clark. Using language. Cambridge University Press,
1996.
20. S. Coradeschi, D. Driankov, L. Karlsson, and A. Saf-
fiotti. Fuzzy anchoring. In IEEE International Confer-
ence on Fuzzy Systems, volume 1, pages 111–114, 2001.
21. S. Coradeschi and A. Saffiotti. Anchoring symbols to
sensor data: Preliminary report. In In Proc. of the 17th
AAAI Conf, pages 129–135. AAAI Press, 2000.
22. S. Coradeschi and A. Saffiotti. An introduction to the
anchoring problem. Robotics and Autonomous Systems,
43(2-3):85–96, 2003.
23. A.M. Cregan. Symbol grounding for the semantic web.
In Lecture Notes in Computer Science (including sub-
series Lecture Notes in Artificial Intelligence and Lec-
ture Notes in Bioinformatics), volume 4519 LNCS,
pages 429–442, 2007.
24. M. Daoutis, S. Coradeschi, and A. Loutfi. Coopera-
tive knowledge based perceptual anchoring. Interna-
tional Journal on Artificial Intelligence Tools, 21(3):44–
87, 2012.
25. M. Daoutis, S. Coradeschi, and A. Loutfi. Towards con-
cept anchoring for cognitive robots. Intelligent Service
Robotics, 5(4):213–228, 2012.
26. M. Daoutis, S. Coradeshi, and A. Loutfi. Grounding
commonsense knowledge in intelligent systems. Jour-
nal of Ambient Intelligence and Smart Environments,
1(4):311–321, 2009.
27. M. Elmogy, C. Habel, and J. Zhang. Multimodal cogni-
tive interface for robot navigation. Cognitive Processing,
12(1):53–65, 2011.
28. S.R.a Fiorini, M.a Abel, and C.M.S.b Scherer. A symbol
grounding model for semantic interpretation of 2-d line
charts. In Proceedings - IEEE International Enterprise
Distributed Object Computing Workshop, EDOC, pages
225–234, 2010.
29. J.F.a Fontanari and L.I.b c Perlovsky. Evolving compo-
sitionality in evolutionary language games. IEEE Trans-
actions on Evolutionary Computation, 11(6):758–769,
2007. cited By (since 1996) 24.
30. J. Fritsch, M. Kleinehagenbrock, S. Lang, T. Pl¨otz,
G. A. Fink, and G. Sagerer. Multi-modal anchoring
for human-robot-interaction. Robotics and Autonomous
Systems, Special issue on Anchoring Symbols to Sen-
sor Data in Single and Multiple Robot Systems, 43(2–
3):133–147, 2003.
31. C. Galindo, J.-A. Fernandez-Madrigal, J. Gonzalez,
A. Saffiotti, and P. Buschka. Life-long optimization of
the symbolic model of indoor environments for a mobile
robot. IEEE Transactions on Systems, Man, and Cy-
bernetics, Part B: Cybernetics, 37(5):1290–1304, 2007.
32. C.a Galindo, A.a Saffiotti, S.a Coradeschi, P.a Buschka,
J.A.b Fernandez-Madrigal, and J.b Gonzalez. Multi-
hierarchical semantic maps for mobile robotics. In
2005 IEEE/RSJ International Conference on Intelli-
gent Robots and Systems, IROS, pages 3492–3497, 2005.
33. A.M. Glenberg and V. Gallese. Action-based language:
A theory of language acquisition, comprehension, and
production. Cortex, 48(7):905–922, 2012.
34. A. Greco and E. Carrea. Grounding compositional sym-
bols: No composition without discrimination. Cognitive
Processing, 13(2):139–150, 2012.
35. D.H. Grollman, O.C. Jenkins, and F. Wood. Discovering
natural kinds of robot sensory experiences in unstruc-
tured environments. Journal of Field Robotics, 23(11-
12):1077–1089, 2006.
36. S. Harnad. The symbol grounding problem. Physica D:
Nonlinear Phenomena, 42(1-3):335–346, 1990.
37. Stevan Harnad. Grounding symbols in sensorimotor cat-
egories with neural networks. In IEE Colloquium (Di-
gest), number 103, pages 6/1–6/4, 1995.
38. F. Heintz, J. Kvarnstr ¨om, and P. Doherty. Stream-
based middleware support for embedded reasoning. In
AAAI Spring Symposium - Technical Report, volume
SS-10-04, pages 59–66, 2010.
39. Fredrik Heintz and Patrick Doherty. Managing dynamic
object structures using hypothesis generation and vali-
dation. In Proceedings of the AAAI Workshop on An-
choring Symbols to Sensor Data, 2004.
40. Fredrik Heintz and Patrick Doherty. Federated dyknow,
a distributed information fusion system for collabora-
tive UAVs. In Proceedings of the International Con-
ference on Control, Automation, Robotics and Vision
(ICARCV), 2010.
41. Fredrik Heintz, Jonas Kvarnstr¨om, and Patrick Doherty.
A stream-based hierarchical anchoring framework. In
Proceedings of IROS, 2009.
42. Fredrik Heintz, Jonas Kvarnstr¨om, and Patrick Doherty.
Bridging the sense-reasoning gap: DyKnow – stream-
based middleware for knowledge processing. Advanced
Engineering Informatics, 24(1):14–26, January 2010.
43. B.H. Hodges, S.V. Steffensen, and J.E. Martin. Caring,
conversing, and realizing values: New directions in lan-
guage studies. Language Sciences, 34(5):499–506, 2012.
44. F. Jamshed and C. Huyck. Grounding symbols: Learn-
ing and grounding 2d shapes using cell assemblies that
emerge from flif neurons. In Proceedings of the Interna-
tional Symposium on AI Inspired Biology - A Sympo-
sium at the AISB 2010 Convention, pages 81–86, 2010.
45. B. Johnston and M.-A. Williams. A formal framework
for the symbol grounding problem. In Proceedings of
the 2nd Conference on Artificial General Intelligence,
AGI 2009, pages 79–84, 2009.
A Short review of Symbol Grounding in Robotic and Intelligent Systems 7
46. B. Johnston, F. Yang, R. Mendoza, X. Chen, and M.-
A. Williams. Ontology based object categorization for
robots. Lecture Notes in Computer Science (including
subseries Lecture Notes in Artificial Intelligence and
Lecture Notes in Bioinformatics), 5345 LNAI:219–231,
2008.
47. D. Jung and A. Zelinsky. Grounded symbolic communi-
cation between heterogeneous cooperating robots. Au-
tonomous Robots, 8(3):269–292, 2000.
48. M. Kandefer and S.C. Shapiro. Knowledge acquisition
by an intelligent acting agent. volume SS-07-05, pages
77–82, 2007.
49. L. Karlsson, A. Bouguerra, M. Broxvall, S. Coradeschi,
and A. Saffiotti. To secure an anchor - a recovery plan-
ning approach to ambiguity in perceptual anchoring. AI
Communications, 21(1):1–14, 2008.
50. J. Kittler, M. Shevchenko, and D. Windridge. Vi-
sual bootstrapping for unsupervised symbol ground-
ing. Lecture Notes in Computer Science (including sub-
series Lecture Notes in Artificial Intelligence and Lec-
ture Notes in Bioinformatics), 4179 LNCS:1037–1046,
2006.
51. O. Kramer. On machine symbol grounding and opti-
mization. International Journal of Cognitive Informat-
ics and Natural Intelligence, 5(3):73–85, 2011.
52. G. M. Kruijff, H. Zender, P. Jensfelt, and H. I. Chris-
tensen. Situated dialogue and spatial organization:
What, where. . . and why? Int. J. of Advanced Robotic
Systems, 4(1):125–138, March 2007.
53. Geert-Jan M. Kruijff, John D. Kelleher, and Nick
Hawes. Information fusion for visual reference resolu-
tion in dynamic situated dialogue. In Perception and In-
teractive Technologies, International Tutorial and Re-
search Workshop, PIT 2006, volume 4021 of Lecture
Notes in Computer Science, pages 117–128. Springer,
2006.
54. K. LeBlanc and A. Saffiotti. Cooperative informa-
tion fusion in a network robot system. In Proc of
the Int Conf on Robot Communication and Coordina-
tion (RoboComm), Athens, Greece, 2007. Online at
http://www.aass.oru.se/˜asaffio/.
55. S. Lemaignan, R. Alami, A. K. Pandey, M. Warnier,
and J. Guitton. Bridges between the Methodological and
Practical Work of the Robotics and Cognitive Systems
Communities - From Sensors to Concepts, chapter To-
wards Grounding Human-Robot Interaction. Intelligent
Systems Reference Library. Springer Publishing, 2012.
56. S. Lemaignan, R. Ros, E.A. Sisbot, R. Alami, and
M. Beetz. Grounding the interaction: Anchoring sit-
uated discourse in everyday human-robot interaction.
International Journal of Social Robotics, 4(2):181–199,
2012.
57. C. Letheby. In defence of embodied cognition: A reply to
fred adams. Phenomenology and the Cognitive Sciences,
11(3):403–414, 2012.
58. W. Lorkiewicz and R.P. Katarzyniak. Issues on aligning
the meaning of symbols in multiagent systems. Studies
in Computational Intelligence, 244:217–229, 2009.
59. A. Loutfi and S. Coradeschi. Improving odour analy-
sis through human-robot cooperation. In Proceedings
- IEEE International Conference on Robotics and Au-
tomation, volume 2005, pages 4443–4449, 2005.
60. A. Loutfi and S. Coradeschi. Smell, think and act: A cog-
nitive robot discriminating odours. Autonomous Robots,
20(3):239–249, 2006.
61. A. Loutfi and S. Coradeschi. Odor recognition for intel-
ligent systems. IEEE Intelligent Systems, 23(1):41–48,
2008.
62. A. Loutfi, S. Coradeschi, M. Daoutis, and J. Melchert.
Using knowledge representation for perceptual anchor-
ing in a robotic system. International Journal on Arti-
ficial Intelligence Tools, 17(5):925–944, 2008.
63. Y. Manabe, S. Fujita, S. Konno, H.a Hara, and K. Sug-
awara. Low-level cognitive process model for symbol
grounding in context-aware system. In Proceedings of
the 10th IEEE International Conference on Cognitive
Informatics and Cognitive Computing, ICCI*CC 2011,
pages 247–253, 2011.
64. Y.a Manabe, S.b Fujitat, S.c Konno, H.a Hara, and K.a
Sugawara. A concept of context-aware computing based
on symbol grounding perspective. In 2010 2nd Interna-
tional Symposium on Aware Computing, ISAC 2010 -
Symposium Guide, pages 86–91, 2010.
65. J. Melchert, S. Coradeschi, and A. Loutfi. Knowledge
representation and reasoning for perceptual anchoring.
In Proceedings - International Conference on Tools with
Artificial Intelligence, ICTAI, volume 1, pages 129–136,
2007.
66. J. Melchert, S. Coradeschi, and A. Loutfi. Spatial re-
lations for perceptual anchoring. In AISB’07: Artificial
and Ambient Intelligence, pages 459–463, 2007.
67. R. Moratz. Intuitive linguistic joint object reference
in human-robot interaction: Human spatial reference
systems and function-based categorisation for symbol
grounding. In Proceedings of the National Conference
on Artificial Intelligence, volume 2, pages 1483–1488,
2006.
68. Yukie Nagai. Learning for joint attention helped
by functional development. Advanced Robotics,
20(10):1165–1181, 2006.
69. Yukie Nagai and Katharina J. Rohlfing. Computational
analysis of motionese: Towards scaffolding robot action
learning. IEEE Transactions on Autonomous Mental
Development, 1(1):44–54, 2009.
70. T. Nakamura, T. Araki, T. Nagai, and N. Iwa-
hashi. Grounding of word meanings in latent dirich-
let allocation-based multimodal concepts. Advanced
Robotics, 25(17):2189–2206, 2011.
71. T. Nakamura, T. Nagai, and N. Iwahashi. Grounding
of word meanings in multimodal concepts using lda. In
2009 IEEE/RSJ International Conference on Intelli-
gent Robots and Systems, IROS 2009, pages 3943–3948,
2009.
72. C.J. Needham, P.E. Santos, D.R. Magee, V. Devin, D.C.
Hogg, and A.G. Cohn. Protocols from perceptual obser-
vations. Artificial Intelligence, 167(1-2):103–136, 2005.
cited By (since 1996) 19.
73. A. Ninio and C. Snow. Pragmatic Development. West-
viewPress, 1996.
74. Daniel Nyga, Moritz Tenorth, and Michael Beetz. Un-
derstanding and executing instructions for everyday ma-
nipulation tasks from the world wide web. In ICRA,
2009.
75. M. Oladell and M. Huber. Symbol generation and
grounding for reinforcement learning agents using af-
fordances and dictionary compression. In Proceedings
of the 25th International Florida Artificial Intelligence
Research Society Conference, FLAIRS-25, pages 132–
135, 2012.
76. K. Pastra. Viewing vision-language integration as a
double-grounding case. In AAAI Fall Symposium -
Technical Report, volume FS-04-01, pages 62–69, 2004.
77. K. Pastra. Vision-language integration: A double-
grounding case. PhD thesis, Department of Computer
Science, University of Sheffield, 2005.
8 Silvia Coradeschi et al.
78. K. Pastra. Praxicon: the development of a grounding re-
source. In Proceedings of the International Workshop on
Human-Computer Conversation, Bellagio, Italy, 2008.
79. K. Pastra. From lexicon to praxicon: language-action-
image semantic relations. In K. Potagas and I. Ev-
dokimidis (eds), Conversations on Language and Ac-
tion, Aiginiteion Series, 2010.
80. K. Pastra, Y. Aloimonos, K. Pastra, and Y. Aloimonos.
The minimalist grammar of action. Philosophical Trans-
actions of the Royal Society B: Biological Sciences,
367(1585):103–117, 2012.
81. K. Pastra and Y. Aloimonos (eds.). Technical Report of
the AAAI 2011 Workshop on ”Language-Action Tools
for Cognitive Artificial Agents: Integrating Vision, Ac-
tion and Language”. AAAI, 2010.
82. K. Pastra, E. Balta, P. Dimitrakis, and G. Karakatsio-
tis. Embodied language processing: a new generation of
language technology. In Proceedings of the AAAI 2011
International Workshop on Language-Action Tools for
Cognitive Artificial Agents: Integrating Vision, Action
and Language, 2011.
83. K. Pastra, P. Dimitrakis, E. Balta, and G. Karakatsi-
otis. Praxicon and its language-related modules. In
Proceedings of Companion Volume of the 6th Hellenic
Conference on Artificial Intelligence (SETN), pages 27–
32, 2010.
84. A. Pieczynska-Kuchtiak. Grounding a descriptive lan-
guage in cognitive agents using consensus methods. In
Lecture Notes in Computer Science, volume 3516, pages
671–678, 2005.
85. M. Raab, M. Wernsdorfer, E. Kitzelmann, and
U. Schmid. From sensorimotor graphs to rules: An agent
learns from a stream of experience. Lecture Notes in
Computer Science (including subseries Lecture Notes
in Artificial Intelligence and Lecture Notes in Bioin-
formatics), 6830 LNAI:333–339, 2011.
86. D. Rodriguez, J. Hermosillo, and B. Lara. Meaning in
artificial agents: The symbol grounding problem revis-
ited. Minds and Machines, 22(1):25–34, 2012.
87. N.P. Rougier. Implicit and explicit representations.
Neural Networks, 22(2):155–160, 2009.
88. L. Schillingmann, B. Wrede, and K. J. Rohlfing. A com-
putational model of acoustic packaging. IEEE Transac-
tions on Autonomous Mental Development, 1(4), 2009.
89. S.C. Shapiro and H.O. Ismail. Anchoring in a grounded
layered architecture with integrated reasoning. Robotics
and Autonomous Systems, 43(2-3):97–108, 2003.
90. L. Steels and T. Belpaeme. Coordinating perceptually
grounded categories through language: A case study for
colour. Behavioral and Brain Sciences, 28(4):469–489,
2005.
91. F. Stramadinoli, M. Rucinski, J. Znajdek, K.J. Rohlfing,
and A. Cangelosi. From sensorimotor knowledge to ab-
stract symbolic representations. In Procedia Computer
Science, volume 7, pages 269–271, 2011.
92. F. Stramandinoli, A. Cangelosi, and D. Marocco. To-
wards the grounding of abstract words: A neural net-
work model for cognitive robots. In Proceedings of
the International Joint Conference on Neural Networks,
pages 467–474, 2011.
93. F. Stramandinoli, D. Marocco, and A. Cangelosi. The
grounding of higher order concepts in action and lan-
guage: A cognitive robotics model. Neural Networks,
32:165–173, 2012.
94. S. Swarup, K. Lakkaraju, S.R. Ray, and L. Gasser. Sym-
bol grounding through cumulative learning. Lecture
Notes in Computer Science (including subseries Lec-
ture Notes in Artificial Intelligence and Lecture Notes
in Bioinformatics), 4211 LNAI:180–191, 2006.
95. M. Taddeo and L. Floridi. Solving the symbol ground-
ing problem: A critical review of fifteen years of research.
Journal of Experimental and Theoretical Artificial In-
telligence, 17(4):419–445, 2005.
96. J. Tani. Symbols and dynamics in embodied cognition:
Revisiting a robot experiment. In Lecture Notes in Ar-
tificial Intelligence (Subseries of Lecture Notes in Com-
puter Science), volume 2684, pages 167–178, 2003.
97. S. Tellex, T. Kollar, S. Dickerson, M.R. Walter, A.G.
Banerjee, S. Teller, and N. Roy. Approaching the symbol
grounding problem with probabilistic graphical models.
AI Magazine, 32(4):64–76, 2011.
98. Moritz Tenorth and Michael Beetz. Towards practical
and grounded knowledge representation systems for au-
tonomous household robots. In ICRA, 2009.
99. M. Tomasello. Constructing language: A usage-based
theory of language acquisition. Cambridge, 2003.
100. M. Vavrecka, I. Farkas, and L. Lhotska. Bio-inspired
model of spatial cognition. Lecture Notes in Computer
Science (including subseries Lecture Notes in Artificial
Intelligence and Lecture Notes in Bioinformatics), 7062
LNCS(PART 1):443–450, 2011.
101. P. Vogt. The physical symbol grounding problem. Cog-
nitive Systems Research, 3(3):429–457, 2002.
102. P. Vogt. Anchoring of semiotic symbols. Robotics and
Autonomous Systems, 43(2-3):109–120, 2003.
103. P. Vogt and E. Haasdijk. Modelling social learning of
language and skills. Artificial Life, 16(4):289–309, 2010.
104. P.A.a Vogt and F.a b Divina. Social symbol grounding
and language evolution. Interaction Studies, 8(1):31–52,
2007.
105. C. Yu and D.H. Ballard. On the integration of grounding
language and learning objects. In Proceedings of the Na-
tional Conference on Artificial Intelligence, pages 488–
493, 2004.
106. H. Zender, P. Jensfelt,
´
O. M. Mozos, G. M. Kruijff, and
W. Burgard. An integrated robotic system for spatial
understanding and situated interaction in indoor envi-
ronments. In In Proc. AAAI ’07, pages 1584–1589, 2007.
