Interactive Technologies for Autistic Children: A Review

Abstract

Recently, there have been considerable advances
in the research on innovative information communication
technology (ICT) for the education of people with
autism. This review focuses on two aims: (1) to provide an
overview of the recent ICT applications used in the treatment
of autism and (2) to focus on the early development
of imitation and joint attention in the context of children
with autism as well as robotics. There have been a variety
of recent ICT applications in autism, which include the use
of interactive environments implemented in computers and
special input devices, virtual environments, avatars and
serious games as well as telerehabilitation. Despite exciting
preliminary results, the use of ICT remains limited. Many
of the existing ICTs have limited capabilities and performance
in actual interactive conditions. Clinically, most
ICT proposals have not been validated beyond proof of
concept studies. Robotics systems, developed as interactive
devices for children with autism, have been used to assess
the child’s response to robot behaviors; to elicit behaviors
that are promoted in the child; to model, teach and practice
a skill; and to provide feedback on performance in specific
environments (e.g., therapeutic sessions). Based on their
importance for both early development and for building
autonomous robots that have humanlike abilities, imitation,
joint attention and interactive engagement are key issues in
the development of assistive robotics for autism and must
be the focus of further research.

Full text

1 23
Cognitive Computation
ISSN 1866-9956
Cogn Comput
DOI 10.1007/s12559-014-9276-x
Interactive Technologies for Autistic
Children: A Review
Sofiane Boucenna, Antonio Narzisi,
Elodie Tilmont, Filippo Muratori,
Giovanni Pioggia, David Cohen &
Mohamed Chetouani

1 23
Your article is protected by copyright and all
rights are held exclusively by Springer Science
+Business Media New York. This e-offprint is
for personal use only and shall not be self-
archived in electronic repositories. If you wish
to self-archive your article, please use the
accepted manuscript version for posting on
your own website. You may further deposit
the accepted manuscript version in any
repository, provided it is only made publicly
available 12 months after official publication
or later and provided acknowledgement is
given to the original source of publication
and a link is inserted to the published article
on Springer's website. The link must be
accompanied by the following text: "The final
publication is available at link.springer.com”.

Interactive Technologies for Autistic Children: A Review
Sofiane Boucenna
•
Antonio Narzisi
•
Elodie Tilmont
•
Filippo Muratori
•
Giovanni Pioggia
•
David Cohen
•
Mohamed Chetouani
Received: 21 May 2013 / Accepted: 29 April 2014
 Springer Science+Business Media New York 2014
Abstract Recently, there have been considerable advan-
ces in the research on innovative information communi-
cation technology (ICT) for the education of people with
autism. This review focuses on two aims: (1) to provide an
overview of the recent ICT applications used in the treat-
ment of autism and (2) to focus on the early development
of imitation and joint attention in the context of children
with autism as well as robotics. There have been a variety
of recent ICT applications in autism, which include the use
of interactive environments implemented in computers and
special input devices, virtual environments, avatars and
serious games as well as telerehabilitation. Despite exciting
preliminary results, the use of ICT remains limited. Many
of the existing ICTs have limited capabilities and perfor-
mance in actual interactive conditions. Clinically, most
ICT proposals have not been validated beyond proof of
concept studies. Robotics systems, developed as interactive
devices for children with autism, have been used to assess
the child’s response to robot behaviors; to elicit behaviors
that are promoted in the child; to model, teach and practice
a skill; and to provide feedback on performance in specific
environments (e.g., therapeutic sessions). Based on their
importance for both early development and for building
autonomous robots that have humanlike abilities, imitation,
joint attention and interactive engagement are key issues in
the development of assistive robotics for autism and must
be the focus of further research.
Keywords Robotics  Children with autism 
Joint attention  Imitation
Introduction
Multimodal social–emotional interactions play a critical
role in child development, and this role is emphasized in
autism spectrum disorders (ASD). In typically developing
children, the ability to correctly identify, interpret and
produce social behaviors (Fig. 1) is a key aspect for com-
munication and is the basis of social cognition [28]. This
ability helps children to understand that other people have
intentions, thoughts and emotions and act as a trigger of
empathy [40, 105]. Social cognition includes the child’s
ability to spontaneously and correctly interpret verbal and
nonverbal social and emotional cues (e.g., speech, facial
and vocal expressions, posture and body movements, etc.);
the ability to produce social and emotional information
(e.g., initiating social contact or conversation); the ability
to continuously adjust and synchronize behavior to others
(i.e., parent, caregivers, peers); and the ability to make an
adequate attribution about another’s mental state (i.e.,
’’theory of mind’’).
ASD are a group of behaviorally defined disorders with
abnormalities or impaired development in two areas: (1)
persistent deficits in social communication and social
S. Boucenna (&)  E. Tilmont  D. Cohen  M. Chetouani
Institut des Systemes Intelligents et de Robotique, CNRS UMR
7222, Universite Pierre et Marie Curie, Paris, France
e-mail: sofiane.boucenna@gmail.com; boucenna@isir.umpc.fr
A. Narzisi  F. Muratori
Division of Child Neurology and Psychiatry, University of Pisa,
Stella Maris Scientific Institute, Calambrone, Italy
E. Tilmont  D. Cohen
Department of Child and Adolescent Psychiatry, AP-HP, Groupe
Hospitalier Piti-Salpltrire, Universit Pierre et Marie Curie, Paris,
France
G. Pioggia
CNR, Rome, Italy
123
Cogn Comput
DOI 10.1007/s12559-014-9276-x
Author's personal copy

interaction and (2) restricted, repetitive patterns of behav-
ior, interests, or activities (http://www.dsm5.org). An
individual with ASD has difficulty interacting with other
people due to an inability to understand social cues. For
example, children with ASD often have difficulty with
cooperative play with other peers; they prefer to continue
with their own repetitive activities [9]. Persons with ASD
evaluate both world and human behavior uniquely because
they react in an abnormal way to input stimuli. They are
problems to engage with human and difficulties to interact
with the environment [120]. Although ASD remain a
devastating disorder with a poor outcome in adult life,
there have been important improvements in treating ASD
with the development of various therapeutic
approaches [33].
Successful autism ‘‘treatments’’ using educational
interventions have been reported as recently as a decade
ago [101]. Since then, the literature devoted to the
description and evaluation of interventions in ASD has
become substantial over the last few years. From this lit-
erature, a number of conclusions can be drawn. First, there
is increasing convergence between behavioral and devel-
opmental methods [107]. For both types of treatment, the
focus of early intervention is directed toward the devel-
opment of skills that are considered pivotal,’’ such as joint
attention and imitation, as well as communication, sym-
bolic play, cognitive abilities, attention, sharing emotion
and regulation. Second, the literature contains a number of
guidelines for treatments [104, 107], such as:
• starting as early as possible
• minimizing the gap between diagnosis and treatment
• providing no shorter than 3/4 h of treatment each day
• involving the family
• providing six-month development evaluations and
updating the goals of treatment
• choosing among behavioral/developmental treatment
depending on the child’s response
• encouraging spontaneous communication
• promoting the skills through play with peers
• gearing toward the acquisition of new skills and to their
generalization and maintenance in natural contexts
• supporting positive behaviors rather than tackling
challenging behaviors.
Toward this direction, ICT may be beneficial in ASD
therapy. Over the last few years, there have been consid-
erable advances in the research on innovative ICT for the
education of people with special needs, such as patients
suffering from ASD [85]. Education is considered to be the
most effective therapeutic strategy [98]. More specifically,
early stage education has proven helpful in coping with
difficulties in understanding the mental states of other
people [73]. In recent years, there have been new devel-
opments in ICT-based approaches and methods for therapy
and the education of children with ASD. Individuals with
autism have recently been included as a main focus in the
area of social signal processing (SSP is the ICT domain
that aims at providing computers with the ability to sense
and understand human social signals and communica-
tion) [30] and affective computing (AC is the ICT domain
that aims at modeling, recognizing, processing and simu-
lating human affects or that relates to, arises from, or
deliberately influences emotions) [32, 51, 82].
In this review, we focus on two aims: (1) to give an
overview of the recent ICT applications that can be used in
the treatment of ASD and (2) to focus on the early devel-
opment of imitation [7, 114, 143
, 151, 155] and joint
attention [49, 119] in the context of children with ASD as
well as robotics.
In ‘‘ICT and autism: an overview’’ section, we describe
the state-of-the-art ICT used in the treatment of ASD. We
show that both the ICT applications and treatment goals are
very different. Regarding the ICT applications, we distin-
guish between interactive environments, virtual environ-
ments, avatars, serious games and telerehabilitation. The
uses of these applications for the treatment of ASD can be
classified according the main goal, which are as follows:
(1) assistive technologies that counteract the impact of
autistic sensory and cognitive impairments on daily life
(close to occupational therapy); (2) cognitive rehabilita-
tion/remediation that attempt to modify and improve the
core deficit in social cognition; and (3) special education
programs for bypassing ASD impairments to help children
acquiring social and academic skills. In ‘‘What is the
contribution of robotics to children with ASD?’’ section,
we focus on robotics and ASD. The robotics platforms are
interesting in the field of interventions in children with
autism because robots generate a high degree of motivation
Fig. 1 Reception and production of social signals multimodal verbal
(speech and prosody) and nonverbal cues (facial expressions, vocal
expressions, mutual gaze, posture, imitation, synchrony, etc.) merge
to produce social signals [30]
Cogn Comput
123
Author's personal copy

and engagement in children with learning disabilities [130]
and can be used to communicate, interact, display and
recognize the ’’emotion’’, develop social competencies and
maintain social relationships [57]. In this section, we focus
on imitation and joint attention from a multidisciplinary
viewpoint to investigate the contribution of social robotics
on children with ASD because these two abilities are
important during the development of the child and for the
robots to be autonomous. We describe the state-of-the-art
solutions proposed in social robotics for imitation and joint
attention. Finally, we focus on the contributions of robotics
on children with ASD and method to evaluate these
architectures.
Many studies have been conducted using very different
technologies that show an interest in a multidisciplinary
research. However, as we will see technologies are limited
in their performance and, from the practical perspective,
they limit the success of experiments with people with
ASD. Moreover, [123]) underlined the unsignificant results
in terms of ’’natural’’ interaction between robots and ASD
individuals. A question as ’’What are the best roles for
robots in therapy?’’ must be addressed to improve the
research quality. In the discussion (‘‘Disscussion and con-
clusion’’ section), we discuss the key issues for improving
ICT devices in the treatment of ASD. We also propose a
new experimental paradigm of learning through imitation
that explores the question of how the robot learning reacts to
different participants (adults, TD children and children with
ASD). This approach allows to analyze and to understand
how cognitive models (as conceptualized through cognitive
computation) are influenced by groups of participants.
ICT and Autism: An overview
Interactive Environments
In recent years, the field of collaborative interactive envi-
ronments, such as virtual environments (VE), has been of
seminal relevance. The advances in this field are the control
of the input stimuli and the monitoring of the child’s
behavior. The aim of interactive computer games is the
improvement of the collaboration between multiple users
such as children with ASD. Moreover, the human–computer
interaction (HCI) is regarded as a safe and enjoyable expe-
rience, which can be explained by the fact that the interaction
with computers, unlike social interactions, does not pose
severe expectations and judgment issues. Therefore, com-
puter systems tend to offer a controlled environment with
minimal distractions, and the use of computers is therefore
attractive for the education of children with ASD [64]. This
finding is further supported by several reports that mention
that this type of interaction elicits positive feelings, whereas
communication with humans could be highly problematic
for children with ASD [74]. Furthermore, tutors often report
that behavioral alterations during the educational process are
a common phenomenon among persons with ASD [79]. The
personal state may be described by specific educational
parameters, such as the time and the processes needed to
complete a goal and the percentage of success. Moreover, the
behavior monitoring during a period of time may reveal
important factors for the children’s progress. A large portion
of the traditional educational tools employs real-world
environments, making the task of educating children with
ASD more difficult [59] because it requires rapid and flexi-
ble thinking. Moreover, real-world environments cannot be
fully controlled because of the inability to provide the same
set of conditions more than one time.
Various interactive environments have been developed
for the rehabilitation of children with autism. In most of the
cases, these environments are introduced through the
means of software education platforms [92, 95]. To pro-
vide knowledge in an attractive way, these platforms use
entertaining content in educational settings (edutainment).
Photographs or sketches of real objects (used in daily life)
are presented on the monitor of a computer to encourage
people with autism to distinguish objects based on their
size, color, type, and so on. Moreover, this type of inter-
active learning platform motivates the children to correlate
the objects with sounds and words. Platforms use animated
pictures or videos to increase the attractiveness of displays.
The comprehension of the task is supported by verbal and
visual (usually makaton
1
symbols) guidance to minimize
the role of the monitoring teacher [89].
The Use of a Computer for Individuals with ASD
Most computer applications designed for people with aut-
ism focus on the relationship between one user and one
computer and aim to help with specific behavioral prob-
lems associated with autism. Authors in [70] claim that
computers are motivating for children with autism due to
their predictability and consistency, compared with the
unpredictable nature of human responses. In regard to
social interaction, the computer does not send confusing
social messages. Research on the use of computers for
students with autism revealed the following [78]:
(a) increase in focused attention; (b) increase in overall
attention span; (c) increase in sitting behavior; (d) increase
in fine motor skills; (e) increase in generalization skills
(from computer to related non-computer activities);
(f) decrease in agitation; (g) decrease in self-stimulatory
1
a very simple language based on a list of simple everyday words,
which uses speech, gesture, facial expression, body language, signs,
symbols and words to aid communication.
Cogn Comput
123
Author's personal copy

behaviors; and (h) decrease in perseverative responses. The
importance of assistive technology for children with autism
has been established by the fact that this technology can be
used in rehabilitation for daily activities. Hetzroni and
Tannous [69] have developed a program (I Can Word It
Too) based on daily life activities in the areas of play, food
and hygiene. The study was conducted on five children
with autism between the ages of 7 and 12, and the focus
was on the effects of using the program on the use of
functional communication. The authors found that use of
the program was effective in improving the communication
of all participants and that the participants were able to
transfer the lessons learned to their natural setting in the
classroom. A DVD with educational software for emotions,
called the Transporters, has been created at the Autism
Research Centre (ARC), which is one of the most exten-
sively used commercial applications for this purpose
(http://www.thetransporters.com/, March, 2009). The
Transporters is based on eight characters, which are vehi-
cles that move according to rule-based motion. Such
vehicles, because of their mechanical nature, attract the
attention of young children with autism. Real-life faces of
actors showing emotions have been grafted onto these
vehicles, and the expressions have been contextualized in
entertaining social interactions between the toy vehicles.
The aim of the Transporters is to determine whether cre-
ating of an autism-friendly context (predictable mechanical
motion) could be learned more easily than is possible in the
real world. The Transporters has been evaluated for
effectiveness in children aged 4–8 with autism. The results
are exciting. (a) In all tasks for which the children were
tested, most children caught up with their typically devel-
oping peers. (b) The results suggest that the Transporters
DVD is an effective way to teach emotion recognition to
children with autism and that the learning generalizes to
new faces and new situations. Children with autism who
did not watch the DVD remained below the typical
developmental levels [62].
Special Input Devices: Touch Screens and Other
Technologies
While people with ASD enjoy interacting with computers,
recent ICT developments allow more attractive forms of
input to be used. In contrast to what has been described in
the previous paragraph, most of the recent research projects
use a touch screen for input feedback instead of a common
mouse device [84]. A multi-user touchable interface that
detects multiple simultaneous touches by two to four users
was used by [61]. Each user sits or stands on a receiver (a
thin pad) such that touching the table surface activates an
array of antennas embedded in its surface (capacitive touch
detection). People with ASD could easily use the screen,
and big colored buttons allow for user selection. Moreover,
studies in using virtual reality (VR) for the rehabilitation of
people with ASD include visual devices that represent the
3D virtual world [137]. Alternative interaction methods
include remote controllers like the Wii-mote (part of a
commercial game console), as demonstrated in [63]. This
device is capable of monitoring not only the remote button
selection but also movements (based on internal acceler-
ometer). Furthermore, external devices are used to measure
and monitor the user’s internal and emotional state, such as
wearable measurement devices [84]. In [139], a web
camera, an eye tracker and a data glove. In addition, sci-
entists have attempted to provide more attractive virtual
worlds by using video projectors and depicting the edu-
cational material on a wall of a room [72]. One of the first
programs to treat children with ASD was TEACCH
(Treatment and Education of Autistic and related Com-
munication handicapped CHildren). TEACCH principles
involve changing the behavior and skill level of the person
based on his or her personal unique needs. In order for a
platform to achieve this goal, it has to be capable of
recording the user’s interaction/education process. By
using all the records in the proper way, a longitudinal
record may be achieved indicating a learning curve for
each autistic person separately, thereby enhancing and
normalizing the educational procedures toward each per-
son’s needs. Consequently, the educators can track each
user’s progress and modify the difficulty levels
accordingly.
Recently, several Apple devices have been used with
ASD patients. Authors in [81] conducted a systematic
review of studies that involved iPods, iPads, and related
devices in teaching programs for individuals with devel-
opmental disabilities including ASD. Fifteen studies cov-
ering the following five domains were examined:
(a) academic, (b) communication, (c) employment,
(d) leisure and (e) transitioning across school settings.
Forty-seven subjects only contributed to these studies
whose aims were (a) delivering instructional prompts via
the iPod Touch or iPad (b) teaching the person to operate
an iPod Touch or iPad to access preferred stimuli. The 15
studies were largely positive and showed that these devices
are viable technological aids for individuals with devel-
opmental disabilities.
Authors in [80] evaluated the effectiveness of a video
modeling package to teach a 5-year-old boy diagnosed with
an ASD basic numeracy skills. The treatment package
consisted of iPad-based video modeling, gradual fading of
video prompts, reinforcement, in vivo prompting and for-
ward chaining. Authors showed clear gains in the partici-
pant’s ability to identify and write the Arabic numerals 1–7
and comprehend the quantity each numeral represents in
association with the lagged intervention. Generalization
Cogn Comput
123
Author's personal copy

and maintenance data demonstrated the robustness of the
treatment effects. This study confirmed that iPad-based
video modeling, when used in a package, can be an
effective technique for teaching numerical skills to children
with an ASD.
Authors in [56] showed that augmentative and alterna-
tive communication (AAC) interventions improve both the
communication and social skills of children with ASD and
other developmental disabilities. New forms of AAC, such
as cell phones, MP3 Players and personal computer tablets,
are explored and evaluated. The authors investigated the
utility of iPad as a communication device by comparing its
use to a communication system using picture cards. Five
school children (6–10 years old) with ASD and develop-
mental disabilities who used a picture card system partic-
ipated in the study. The results were mixed in that
communication behaviors either increased when using the
iPad or remained the same as when using picture cards.
Recently, [100] used an iPad play story to increase the
pretend play skills in 4 preschoolers with ASD. The story
utilized a series of video clips depicting toy figures,
engaged in a pretend play vignette, producing scripted
character dialogue. Three of the participants demonstrated
increases in the target behavior (the play dialogue), and the
effects were largely maintained during generalization
opportunities with peers and during a 3-week follow-up
condition.
Virtual Environments
Virtual environments (VEs) have proven to be another
active area of research for social interventions for autistic
children [13]. Various successful software platforms with
virtual environments for autistic people have been devel-
oped over the last decade [48, 50]. VE are able to mimic
specific social situations in which the user can participate
in role play. The stable and predictable environment pro-
vides types of interaction that eliminate the anxiety [112].
Moreover, VE offer safe, realistic-looking 3D scenarios
that can be built to depict everyday social scenarios. The
use of animation is also in line with research indicating that
children with learning disabilities prefer programs that
include animation, sounds and voices [146].
Recent works have demonstrated the ability of partici-
pants with ASD to use and to interpret VE successfully and
to learn simple social skills using the technology [112, 113,
137]. Additionally, one of the most important aspects of
VE used by participants with ASD in educational settings
is the participants level of enjoyment. Persons with ASD,
especially children, are more interested in interacting with
computers than other toys [85]. Moreover, virtual
peers [141] are life-sized, language enabled, computer-
generated and animated characters that look like a child,
which are capable of interacting, sharing real toys and
responding to children’s input. For example, a virtual peer
accompanies a child with ASD during a game or a storing
telling scenario. A number of researchers have developed
interesting research contributions using storing telling
scenarios. For example, [99] developed and tested a virtual
cafe for children with autism to address impairments in
social interaction. The participants were required to per-
form specific tasks in the virtual cafe, such as ordering and
paying for a drink and finding a place to sit. Again, navi-
gation was achieved through the use of a mouse. A virtual
reality social-understanding training program was admin-
istered to six adolescents, 14–16 years old, each with for-
mal diagnoses of an autism spectrum disorder. During the
training sessions, four types of activities were taught and
practiced. These activities were graded in difficulty and
created based on certain social conventions associated with
finding a seat in an empty or crowded cafe, ordering,
paying and engaging in appropriate conversation with
others. The social understanding of these adolescents was
assessed using ratings of their verbal descriptions of their
decision-making process of how they would behave in two
different social scenarios, which were: a cafe and a bus.
The former was similar to situations encountered in the
virtual cafe, while the latter assessed the generalizability of
the participants’ learned social understanding. The results
were variable and only two participants showed gains in
social knowledge in both scenarios. Actual performance in
real situations was not assessed. Because real–cafe inter-
actions usually require touching objects, such as money or
coffee mugs, the integration of more complex haptics into
this type of program may facilitate more realistic interac-
tion between the user and VE.
Increased realism [11, 67] would influence the degree of
ecological validity achieved and subsequent degree of skill
transfer. Increasing in complexity, touch screen technology
has facilitated human-computer interaction without a tra-
ditional mouse or joystick. Authors in [67] created a virtual
supermarket on a flat screen monitor to teach 2 children, 8
and 15 years old, how to think abstractly and play imagi-
natively. The children explored the virtual supermarket
through touching the screen. They interacted with the
objects in increasingly more imaginative ways, such as
transforming a pair of flying pants into a highway. The
authors assessed the outcomes using a test of functional
object use (i.e., how an object should be used), the sym-
bolic play test (SPT) (1976), the test of pretend play (ToPP)
(1997) and the Imagination and magic understanding tests.
Children improved on all tests except the SPT. The authors
concluded that their virtual reality tool is useful in
improving the symbolic thinking skills of these children
and that these skills translate into concrete symbolic play
behaviors. The touch screen facilitated easy interaction
Cogn Comput
123
Author's personal copy

between the children and the display interface and allowed
the instructor to participate as well. This multidimensional
interaction is naturally afforded by touch screen technol-
ogy; touch screen technology allows for interaction
between the child and computer, instructor and computer,
and instructor and child.
Diamond Touch (Circle Twelve Inc., Framingham,
Mass., USA), a state-of-the-art multi-user and multi-touch
display table, allows many people to interact with objects
on the table-top display screen simultaneously through
touch. Similar to the touch screen in [67], the Diamond
Touch table immerses users in an imaginative scene where
their actions and decisions have real-time consequences
within the virtual world. Diamond Touch technology was
integrated with the Story Table interface to allow multiple
children to create an imaginative story together by select-
ing, combining and sequencing a series of on-screen virtual
characters and events. Some story elements required two
children to touch the screen before they were integrated
into the story, reinforcing joint attention, communication
and negotiation. Authors in [11] evaluated this system with
three dyads (a dyad is composed of two children with
autism), aged 9–11 years old, to teach and reinforce key
social skills, such as eye contact, turn-taking, sharing and
joint directed behavior. During the intervention, the dyads
were instructed to create and narrate stories using back-
grounds and characters that were jointly chosen. The
instruction was focused on three goals, which were: per-
forming shared activities, helping and encouraging each
other and persuading and negotiating when creating the
stories. Ratings of social behaviors from the videos of the
Story Table sessions were completed; in addition, the
authors assessed the generalizability of the children’s
social skills through a Legolike assembly game, Marble
Works. After the training sessions, the children were all
rated as having more positive social behaviors during the
use of the Story Table and more positive behaviors during
the use of Marble Works. In addition to the improvements
in the positive social behaviors, the quality of play of the
dyads improved from simple parallel play without eye
contact to complex, coordinated play. The authors con-
cluded that the Story Table intervention increased both the
quantity and the quality of social interaction between the
dyads.
Both [67] and [11] provide evidence that touch screen
technology shows great promise in promoting creative and
imaginary play between multiple users. Authors in [152]
highlight that future studies should consider using typical
peers, rather than atypical peers, as participants with this
multiuser technology. In fact, research has shown that
same-aged, typical peers serve as effective role models for
children with autism to reinforce prosocial and age
appropriate behaviors [46]. It is important to note that
although devices such as the mouse, joystick and touch
screen cannot simulate real-life haptic interactions, such as
feeling the texture of a surface, incorporating the sense of
touch adds yet another layer of interaction within the
program. Participating in real-time cause-and-effect
behaviors may contribute to an overall sense of presence
and motivation for the child during the intervention
program.
Avatars for Autism and Serious Games
Playing, in most cases, an essential role as the instructor,
emotionally expressive avatars are among the most inter-
esting options of the educating system. The current litera-
ture reveals that avatars, humanoid or not, advance the
educational process [85] and improve the social skills of
the participants [71]. Additionally, educators suggest that
most of the time, persons with ASD can recognize the
avatar’s mental and emotional state from the facial
expressions [85, 106]. Avatars, as inhabitants of the virtual
space, can enhance the interaction level in VE. Their
behavioral capabilities are envisaged with emotional and
facial expressions [52]. The use of emotionally expressive
avatars is of crucial importance in the educational process
because their ability to show emotions and empathy
enhances the quality of tutor–learner and learner–learner
interaction [53]. Therefore, emotionally aware computers
are regarded as a considerable and valuable educational
technique [120]. A significant effort has been undertaken
to use emotionally avatars due to the findings in psychol-
ogy and neurology that suggest emotions are an important
factor in decision-making, problem solving and cognition
in general [35]. The results of surveys among educators of
autistic children in the recent literature illustrate that the
children recognition of not only the avatars emotion but
also the avatar’s emotional state advances the educational
process [85]. Moreover, the findings are better in the case
of avatars that have voices [85]. Apart from the instructor
form, the avatar is responsible for providing feedback to
the user’s action by means of the appropriate emotion
(happy for success and sad for failure). Training studies
in [126] have suggested that children with autism show
greater improvements in emotion recognition when pro-
grams include cartoons rather than photographs of real
faces [12]. Moreover, clinical and parental reports also
state that autistic children spend long periods of time
looking at cartoons [126]. Additionally, parents and pro-
fessionals often report that ‘‘autistic children know more
about cartoons than about people’’ [126].
Recently, [132] developed a serious game, ’’Jestimule’’,
to improve social cognition in ASD. The authors attempted
to develop the game with consideration for the heteroge-
neity of ASD. ICT was also used to facilitate the use of the
Cogn Comput
123
Author's personal copy

game by young children or by children with developmental
delays (e.g., haptic joystick for feedback). They also
evaluated the serious game for its effectiveness in teaching
ASD individuals to recognize facial emotions, emotional
gestures and emotional situations (Fig. 2). First, they
showed that a group of 40 individuals (aged from 6 to 18)
who used ’’Jestimule’’ at the hospital twice a week 1 h for
4 weeks of exploration could play and understand the
serious game even when they had comorbid intellectual
disabilities. They also showed that participants improved
their recognition of facial emotions, emotional gestures and
emotional situations in different tasks. These preliminary
results have clear education and therapeutic implications in
ASD and should be taken into account in future training.
Telerehabilitation for Autism
Telerehabilitation is an emerging method of delivering
rehabilitation services that uses technology to serve clients,
clinicians and systems by minimizing the barriers of dis-
tance, time and cost. More specifically, telerehabilitation
can be defined as the application of telecommunication,
remote sensing and operation technologies, and computing
technologies to assist with the provision of medical
Fig. 2 Main principles of the serious games JeStimule. The games
are divided into three phases. The learning phase is composed of a
series of games with increasing complexity. The subjects will learn to
recognize the facial (screen 1) and gesture (screen 2) emotions of
avatars. During the practice phase, the child plays in a virtual
environment and circulates in five different areas of life, which are:
square (screen 3), theater (screen 4), restaurant, garden and store. The
participant should recognize or anticipate the expression of the
emotional avatars in various social situations using the learning
undertaken in the first phase. Interestingly, the game is adaptable for
individuals with low and high functioning because, for the same
social situation, it is possible to choose the best response modality
appropriate to the cognitive skills of players. The modalities include a
color code mode (screen 5) for non-readers, emotional words (screen
6) for readers or idiomatic expressions (screen 7) for individuals with
Asperger syndrome. For each social situation, the player should
recognize or anticipate emotions. If the answer is correct, the player
wins a puzzle piece and makes a choice of action. He or she must
make a choice of action by selecting one of four proposed actions
with a pictogram, which are: stay put, run away, assist or cheer
(screen 8). If the answer is incorrect, the player does not win a puzzle
piece and visualizes the scene again later. Then, the player runs a the
gaming platform again to meet a new social scene. The goal of the
game is very simple; the player wins a piece of a puzzle for each
emotion recognized or anticipated. At the end of the module of the
game, he or she has won 30 pieces. (courtesy of Sylvie Serret and
Florence Askenasi [132])
Cogn Comput
123
Author's personal copy

rehabilitation services at a distance. Much attention has
been paid to the efficacy of telerehabilitation in the effort to
decrease the time and cost associated with the delivery of
rehabilitation services.
Some studies have also compared telerehabilitation
services with face-to-face interventions to evaluate whether
these approaches are ‘‘as good as’’ traditional rehabilitation
approaches. However, telerehabilitation may in fact pro-
vide new opportunities that are more effective by increas-
ing accessibility and creating the least restrictive
environment. Technologies that enable telerehabilitation
services, such as increased computer power and the
availability of high-speed data transmission lines, have
become more prominent in recent years [43]. Winters
provides a comprehensive review of the conceptual models
of telerehabilitation [156]. Authors in [43] explain that
telerehabilitation falls under a broader category of services
that use telecommunication to provide health information
and care across distances, termed telehealth. Telehealth is
broken into 3 subcategories, which are: telemedicine,
telehealthcare and e-health/education. Most of the research
literature on telerehabilitation has focused on outcomes
measures for decreasing costs, saving travel time and
improving access to specialty services and expert practi-
tioners [10]. The rationale proposed to support the explo-
ration and implementation of telerehabilitation has been
essentially based on the use of various technologies to
address geographic and economic barriers and potentially
enhance cost-effectiveness. There is also a significant
impetus to support the value of medical rehabilitation
services delivered in the home. Although much of this
literature seems to be motivated by providing a rationale
for expeditious discharge from the inpatient setting for
cost-saving purposes, the research supports that the deliv-
ery of some home-based rehabilitation services is at least as
effective as the delivery of those services in hospitals. In
some cases, telerehabilitation adds contextual factors that
enhance rehabilitation and outcomes. These findings sup-
port the development and implementation of telerehabili-
tation approaches to facilitate naturalistic rehabilitation
treatment in the home. Intervention in the home or work
environment has been provided remotely for numerous
needs, including cognitive rehabilitation using the Internet,
constraint-induced movement therapy using a computer
and sensors to guide the patient through exercises [91] and
speech pathology for children with autism [111].
An interesting contribution is the use of telerehabilita-
tion in children with autism. A number of researchers at the
UC Davis MIND Institute are examining technology tools
that will enable families to interact from their own homes
with therapists and receive ‘‘long distance’’ guidance for
interventions with their children [149]. At present, there
are various challenges to delivering health care to families
with ASD, such as long waiting lists and few specialist
services. Barriers to service delivery and utilization are
additionally exacerbated for families living in rural or
remote areas, often resulting in limited access to preven-
tative mental health services in general and parenting ASD
interventions in particular. Telecommunication technology
can support long-distance clinical health care; however,
there is little information as to how this resource may
translate into practice for families with ASD. The Vismara
study examined the use of telemedicine technology to
deliver a manualized, parent-implemented intervention for
families of children with ASD, ages 12–36 months. It was
hypothesized that telemedicine technology as a teaching
modality would optimize parenting intervention strategies
for supporting children’s social, affective, communicative
and play development. Recruited families received 12
weekly 1-h sessions of direct coaching and instruction of
the early start denver model (ESDM) [38]. A parent
delivery model was provided through an Internet-based
video conferencing program. Each week, parents were
coached on a specific aspect of the intervention through a
video conferencing program and webcam, allowing the
parent and therapist to see, hear and communicate with one
another. Parents were taught how to integrate the ESDM
into natural, developmentally and age appropriate play
activities and caretaking routines in their homes. Video
data were recorded from 10 min of parent–child interaction
at the start of each session and coded by two independent
raters blinded to the order of the sessions and hypotheses of
the study. The preliminary findings of this study suggested
that integrating telemedicine as a teaching modality
enabled the following: (a) parents to implement the ESDM
more skillfully after coaching and (b) an increase in the
number of spontaneous words, gestures and imitative
behaviors used by the children. The current findings sup-
port the efficacy and cost-effectiveness of using telemedi-
cine to transfer a developmentally based, relationship
focused and behaviorally informed intervention (i.e., the
ESDM) into parents’ homes to be delivered within typical
parent-child activities. Additional research is needed to
confirm the promise and utility of telemedicine for trans-
porting services to families with limited access.
Conclusion
ICT-based approaches and methods are used for the ther-
apy and special education of children with ASD. ICT
research has explored several approaches for the treatment
of persons with ASD, which are: (1) counteracting the
impact of autistic sensory and cognitive impairments on
daily life (assistive technologies, e.g., [100]); (2) trying to
modify and improve the core deficit in social cognition
(cognitive rehabilitation/remediation, e.g., [132]); and (3)
Cogn Comput
123
Author's personal copy

bypassing ASD impairments to help children acquire social
and academic skills (special education, e.g., [89]). How-
ever, much has yet to be improved to attain significant
success in treating individuals with ASD. From the prac-
tical perspective, many of the existing technologies have
limited capabilities in their performance, which limits the
success of ICT treatment in persons with ASD. Clinically,
most ICT proposals have not been validated outside the
context of proof of concept studies. Because most ICTs
have limitations (e.g., the interaction is not natural, intui-
tive or physical), emerging research in the field of autism is
aimed at the integration of social robotics [44, 86, 154].
Social robots are used to communicate, display and rec-
ognize the ’’emotion’’ and develop social competencies and
maintain social relationships [57]. Developed as interactive
toys for children, humanoid robots are used as research
platforms for studying how a human can teach a robot,
using imitation, speech and gestures. Increasingly, robotic
platforms are developed as interactive playmates for chil-
dren. Recent literature reveals that robots generate a high
degree of motivation and engagement in children with
learning disabilities, especially in persons with ASD,
including those who are unlikely or unwilling to interact
socially with human educators and therapists [130]. In the
next section, we will show how social robots can improve
or help better understanding the condition of children with
ASD.
What is the Contribution of Robotics to Children
with ASD?
In this section, we explore the contribution of robotics to
children with ASD. The use of robots in special education
is an idea that has been studied [108]. We will specifically
focus on robotics and children with ASD according to what
is expected from the robotics in the context of the specific
experiment described. However, it is important to keep in
mind that socially assistive robotics have at least three
discrete but connected phases, which are: physical robot
design, human–robot interaction design and evaluations of
robots in therapy-like settings [131]. Moreover, we focus
on two abilities, imitation and joint attention because they
are important during the development of the child [29, 76,
77, 145] and core deficit in ASD [38]. To address these
abilities from the point of view of both developmental
psychology and ICT, we begin by briefly describing the
different architectures developed in robotics for imitation
and joint attention. Next, we review the available literature
on robotics and ASD, differentiating between different
lines of research, including: (1) exploring the response of
children with ASD to robotics platforms, (2) settings where
a robot was used to elicit behaviors, or (3) modeling or
teaching a skill and last (4) providing feedback to children
with ASD.
Robot Imitation Skills
Beginning with Kuniyoshi’s studies [6, 88], learning by
observation has been shown to proceed in three phases: (1)
observation, which is watching an action performed by a
human, e.g., a human grasps an object and then moves it to
another position; (2) understanding, which involves the
construction and memorization of an internal representa-
tion of the observed task; and (3) reproduction of the
observed task. This approach has been used in several
studies in different contexts, such as household environ-
ments [45], labyrinths [66] and learning sequences [15,
16]. In other studies, imitation has been used to reproduce
an observed gesture (i.e., a low-level gesture).
Several research questions are thus centered on move-
ment recognition (can the robot identify the human arm and
characterize the human arm trajectory?), the form of the
gesture (what should the robot imitate?) and the perspec-
tive being considered. A solution to the last issue might be
to perform the gestures directly with a robotic forelimb,
e.g., using a remote control [26] to manipulate the
hand [25] or by fitting a robot model with sensors [
1, 96]
or an exoskeleton [75]. Moreover, [127] show that the
coupling of perception and action processes plays an
important role in basic capabilities of social interaction.
They attempt to endow artificial embodied agents with
similar abilities, and they present a probabilistic model for
the integration of perception and generation of hand-arm
gestures via a hierarchy of shared motor representations.
Imitation that involves interaction with the environment
is more complex. The difficulty is in determining the
relationships among the hands, arms and different objects.
However, humans can aid the robot by specifying the
relationships among the objects. The robot can also be
endowed with primitive movements such as grasping an
object. These primitive movements provide a vocabulary of
actions for the robot, which the robot must then learn to
combine to perform complex tasks [109, 110]. However,
an important limitation is that the robot can only learn to
perform tasks that require this primitive repertoire. Con-
sequently, robotics with these designs have developed the
following capabilities: (1) learning primitive movements
such as grasping an object [26] or putting it in a box [68]
and (2) performing gestures by adapting to the environ-
ment [26, 65, 135].
Authors in [22] investigated how robots learn to rec-
ognize facial expressions without having a teaching signal,
allowing the robots to associate facial expressions with
given abstract labels (e.g., the name of the facial emotional
expressions for ‘‘sadness,’’ ‘‘happiness,’’ etc.). The authors
Cogn Comput
123
Author's personal copy

also developed a sensory motor architecture for the rec-
ognition of facial expressions. The robot can learn facial
expressions if it produces these facial expressions, and the
human imitates the robot’s facial expression to facilitate
online learning (Fig. 3 shows the human–robot interaction
game).
These authors showed in their first series of robotics
experiments that a simple neural network model could
control the robot’s head, and the robot could learn online to
recognize facial emotional expressions (the human partner
imitated the robot’s prototypical facial expressions). Imi-
tation was used as a communication tool instead of a
learning tool; the caregiver communicated with the robot
through imitation. Moreover, the same architecture could
be used to learn posture recognition [21] and joint
attention [24].
Figure 4 shows an interaction between the child with
ASD and the robot during an imitation game. In the first
phase of the interaction (learning phase), the robot pro-
duces postures and we ask to the child to mimic the robot
posture. After this first phase, which lasts for approxi-
mately 2 min, the robot must mimic the postures of the
child with ASD. Currently, we perform the learning of
posture with children with ASD to show that the robot is
able to learn this task with children with ASD. Moreover,
we analyze the influence of the partners (children with
ASD, typical children and adults) who interact with the
robot during this imitation game [20].
Others studies have also proposed neural network
architectures designed to exhibit learning and communi-
cation capabilities via imitation [3, 4, 39]. An artificial
system does not need to incorporate any other internal
model to perform real-time and low-level imitations of
human movements despite the related correspondence
problem between humans and robots. A simple sensory
motor architecture can perform such tasks. These sensory
motor architectures and this type of paradigm are inter-
esting because robots are able to learn online and autono-
mously, which allows for the creation of a real interaction
between a human partner (e.g., a child) and robot. In this
case, the human partner communicates with the robot
through imitation.
(a)
(b)
Fig. 3 Experimental protocol. a In the first phase of the interaction,
the robot produces a random facial expression (sadness, happiness,
anger or surprise) plus a neutral face for 2 s; then, the robot returns to
a neutral face for 2 s to avoid human misinterpretation of the robot
facial expression (the same procedure is used in psychological
experiments). The human subject is asked to mimic the robot head.
b After this first phase, which lasts between 2 and 3 min according to
the subject’s ’’patience’’, the generator of the random emotional states
is stopped. If the neural network has learned correctly, then the robot
must mimic the facial expression of the human partner [23]
Fig. 4 Example of typical human–robot interaction. In this case, the
child with ASD imitates the robot [21]
Cogn Comput
123
Author's personal copy

Are Robots Able to Develop Joint Attention
Capabilities?
The joint attention is essential for social interaction and
for building robots that can interact in social environments,
as successfully implemented using the Baron–Cohen
model [8] and a humanoid robot [128, 129]. According to
Baron-Cohen, joint attention is based on the following two
modules: (1) the intentionality detector (ID), which uses
sensory modalities and can interpret the actions of other
agents, for example, purpose, goal and desire; and (2) the eye
direction detector (EDD), which can detect the presence and
gaze direction of other agents. EDD allows the robot to infer
that a person is looking at an object if his/her gaze is directed
toward that object. ID allows for the interpretation of the
gaze direction as a goal state and the interpretation of the
gaze of others as intentions. More recently, other proposals
have been made. The model developed by [157] implements
an effective model, which integrates image-processing
algorithms into a robust estimation of the head pose and an
estimation of the gaze direction. Other authors, such as [93,
94, 134], have focused on the capacity of shared attention in
‘‘mental rotation’’ and ‘‘perspective taking.’’ These capa-
bilities allow the humanoid robot HRP2 to acquire repre-
sentations of the environment from other perspectives and to
assimilate the concept of reason from the perspectives of
others to obtain a representation of the knowledge of others.
Nagai [103] proposed a developmental model, which
would allow a robot to acquire joint attention capability
without the assessment of the task. This model showed how
a robot could interpret the gaze direction of humans to
focus on objects in the environment. The robot acquired the
ability of joint attention without any task evaluation from a
human caregiver. Moreover, the robot attempted to repro-
duce the staged developmental process of infant joint
attention. In another study, joint attention can emerge from
a sensory motor architecture [24]. In summarizing the
challenges of joint attention, [83] attempted first to define
this mechanism as well as the unitary elements that con-
stitute it. In line with Tomasello’s views, [83, 144] argued
that joint attention implies viewing the behavior of other
agents as intentionally driven. In that sense, joint attention
is much more than gaze following or simultaneous looking.
Robotics and Children with Autism
Since 2000, there have been an increasing number of
clinical studies that have used robots to treat individuals
with ASD. The robot can have two roles in the interven-
tion, which are practice and reinforcement [47]. At least
three reviews of the literature have been conducted
recently [44, 131, 142]. Here, we choose to follow the plan
proposed by Diehl et al. because it fits the main focus of
our study regarding imitation and joint attention. Diehl
et al. distinguished four different categories of studies. The
first compares the responses of individuals with ASD to
humans, robots or robotlike behavior. The second assesses
the use of robots to elicit behaviors that should be pro-
moted with regard to ASD impairments. The third uses
robotics systems or robots to model, teach and practice a
skill with the aim of enhancing this skill in the child. The
last uses robots to provide feedback on performance during
therapeutic sessions or in natural environments.
Response to Robots or Robotlike Characteristics
Although most of the research in this field has been based
on short series or case reports, the authors have insisted on
the appealing effects of using robots to treat individuals
with ASD. If we assume that individuals with ASD prefer
robots or robotlike characteristics to human characteristics
or non-robotic objects, we may wonder why individuals
with ASD prefer robots as well as what is particularly
appealing about these characteristics. Authors in [117]
compared a child with ASD to a typically developing
control child for his/her behavioral and physiological
responses to a robotic face. The child with ASD did not
have an increase in heart rate in response to the robotic
face, which implies that the robotic face did not alarm the
child. In contrast, the control child spontaneously observed
the robot with attention and expressed positive reactions to
it; however, when the robot’s facial movements increased,
the typical child became uncomfortable and exhibited an
increased heart rate. In a case series, the same author [118]
compared the responses of ASD children to the robotic face
versus human interaction; most individuals with ASD
showed an increase in social communication, some showed
no change and one showed a decrease when he interacted
with the robotic face Fig. 5.
Authors in [
55] showed in a group of eight children with
ASD that there was tremendous variability in the valence
of an effective response toward a mobile robot, depending
on whether the robot’s behavior was contingent on the
participant or random. In this study, the robot automatically
distinguished between positive and negative reactions of
children with ASD. Individual affective responses to the
robots were indeed highly variable. Some studies [37, 124]
have shown that for some children with ASD, there is a
preference for interacting with robots compared to non-
robotic toys or human partners. However, [37] found
individual differences in whether children with ASD pre-
ferred robots to non-robotic toys. Two of the four partici-
pants exhibited more eye gazes toward the robot and more
physical contact with the robot than with a toy.
Other studies have investigated movements. Authors
in [17] found a speed advantage in adults with ASD when
Cogn Comput
123
Author's personal copy

imitating robotic hand movements compared to human
hand movements. In the same vein, [116] reported that
children with ASD made significantly faster movements to
grasp a ball when they observed a robotic arm perform the
movement compared to a human arm. In contrast, typically
developing children showed the opposite effect. Therefore,
these two studies suggest increased imitation speed with
robot models compared to human models [17, 116].
Additionally, some studies have investigated the
responses of children with ASD when exposed to emo-
tional stimuli. Authors in [102, 133] explored the responses
of 3- and 5-year-old children to emotional expressions
produced by a robot or a human actor. Two types of
responses were considered, which were: automatic facial
movements produced by the children facing the emotional
expressions (emotional resonance) and verbal naming of
the emotions expressed (emotion recognition). Both studies
concluded that, after robot exposition, an overall increase
in performance occurred with age, as well as easier rec-
ognition of human expressions [102, 133]. This result is
encouraging from a remediation perspective in which an
expressive robot could help children with autism express
their emotions without human face-to-face interaction.
Finally, [31] investigated the neural bases of social inter-
actions with a human or with a humanoid robot using fMRI
and compared male controls (N = 18, mean age = 21.5
years) to patients with high-functioning autism (N = 12,
mean age = 21 years). The results showed that in terms of
activation, interacting with a human was more engaging
than interacting with an artificial agent. Additionally, areas
involved in social interactions in the posterior temporal
sulcus were activated when controls, but not subjects with
high-functioning autism, interacted with a human fellow.
Robots can be Used to Elicit Behavior
Some theoretical works have highlighted several potential
uses of a robot for diagnostic purposes [130, 140]. For
example, a robot could provide a set of social cues
designed to elicit social responses for which the presence,
Fig. 5 As shown in screen 5a, the cardiac frequency of the patient
increases after his attention is focused on the robot and remains fairly
even until he is forced to focus on his emotional relationship with
FACE. In screen 5b, the subject is shown to completely focus his
attention to FACE; in screen 5c, the subject is spontaneously making
eye contact with FACE; screen 5d shows the nonverbal request of the
subject through a conventional gesture (a wink) (from [118])
Cogn Comput
123
Author's personal copy

absence or quality of response is helpful during diagnostic
assessment. In [54], the robot could be programmed to take
the role of a bubble gun
2
. The robot produces bubbles to
elicit an interaction between the child and the examiner.
Additionally, the robot can act as a sensor and provide
measurements of targeted behaviors [130, 140]. These
measurements may be used to diagnose the disorder and to
quote its severity on one or several dimensions. The robots
could record behaviors and traduce social behaviors into
quantitative measurements. Additionally, interaction
between a robot and a child has been used to elicit and
analyze perseverative speech in one individual with high-
functioning ASD [136]. Interaction samples were collected
from previous studies in which the child interacted with a
robot that imitated the child’s behavior. Here, the robot–
child interaction is used to collect samples of perseverative
speech to conduct conversational analysis on the inter-
changes. This study suggested that robot–child interactions
might be useful to elicit characteristic behaviors such as
perseverative speech.
Finally, the robot can be used to elicit prosocial
behaviors. Robots can provide interesting visual displays or
respond to a child’s behavior in the context of a therapeutic
interaction. Consequently, the robot could encourage a
desirable or prosocial behavior [36, 54]. For example, the
robot’s behavior could be used to elicit joint attention; first,
the robot could be the object of shared attention [36] or the
robot could provoke joint attention by looking elsewhere at
an object in the same visual scene and asking the child with
ASD to follow its gaze or head direction. In another
study, [122] showed that individuals with ASD are able to
follow social referencing behaviors performed by a robot.
This study shows that social referencing is possible, but the
results are not quantitative. Other studies [58, 125] have
tried to elicit prosocial behavior, such as joint attention and
imitation. However, the results were not robust because of
the small sample size of children with ASD in these
studies. Finally, several studies aimed to assess whether
interaction between a child with ASD and a robot with a
third interlocutor can elicit prosocial behaviors [34, 87,
150]. Unfortunately, no conclusion could be drawn due to
their small sample sizes and the significant individual
variation in the response to the robot.
Robots can be Used to model, Teach or Practice a Skill
Here, the theoretical point of view is to create an envi-
ronment in which a robot can model specific behaviors for
a child [36] or the child can practice specific skills with the
robot (Scassellati speaks out ’’social crutch’’, [130]). The
aim is to teach a skill that the child can imitate or learn and
eventually transfer to interactions with humans. In this
case, the robot is used to simplify and facilitate social
interaction. The objective of Duquette [47] was to explore
whether a mobile robot toy could facilitate reciprocal social
interaction in cases in which the robot was more predict-
able, attractive and simple. The exploratory experimental
setup presented two pairs of children with autism, a pair
interacting with the robot and another pair interacting with
the experimenter. The results showed that imitations of
body movements and actions were more numerous in
children interacting with humans compared to children
interacting with the robot. In contrast, the two children
interacting with the robot had better shared attention (eye
contact and physical proximity) and were better able to
mimic facial expressions than the children interacting with
a human partner. [60] used techniques for mimicking and
evaluating human motions in real time using a therapeutic
humanoid robot. Practical experiments have been per-
formed to test the interaction of ASD children with robots
and to evaluate the improvement of children’s imitation
skills.
Robots can be Used to Provide Feedback
and Encouragement
Robots can also be used to provide feedback and encour-
agement during a skill learning intervention because indi-
viduals with ASD might prefer the use of a robot than a
human as a teacher for skills. Robots can have humanlike
characteristics. For example, they can mimic human
sounds or more complex behaviors. The social capabilities
of robots could improve the behavior of individuals with
ASD vis—vis the social world. The robot could also take
on the role of a social mediator in social exchanges
between children with ASD and partners because robots
can provide feedback and encouragement [36]. In this
approach, the robot would encourage a child with ASD to
interact with an interlocutor. The robot would provide
instruction for the child to interact with a human therapist
and encourage the child to proceed with the interaction.
However, this approach is only theoretical, as no studies
have yet been conducted.
However, some attempts at using robots for rewarding
behaviors have been made. [47] used a reward in response
to a robot behavior. For example, if a child was successful
in imitating a behavior, the robot provided positive rein-
forcement by raising its arms and saying, ‘‘Happy.’’
Additionally, the robot could respond to internal stimuli
from the child; for example, the stimuli generally used in
biofeedback (e.g., pulse and respiratory frequency) could
be used as indicators of the affective state or arousal level
2
When the child pushes one of the buttons, the robot blows bubbles
while turning in place. When the child does not push one of the
buttons, the robot does nothing (no bubbles, no turning).
Cogn Comput
123
Author's personal copy

of the child to increase the individualized nature of the
treatment [115]. This capability could be useful to provide
children with feedback about their own emotional states or
to trigger an automatic redirection response when a child
becomes disinterested [90].
Discussion and Conclusion
Recent years have witnessed ICT-based approaches and
methods for the therapy and education of children with
ASD. Individuals with autism have lately been included as
a main focus in the area of AC [82]. Technologies, algo-
rithms, interfaces and sensors that can sense emotions or
express them and thereby influence the users’ behavior
(here individuals with ASD) have been continuously
developed. Working closely with persons with ASD has led
to the development of various significant methods, appli-
cations and technologies for emotion recognition and
expression. Innovative wearable sensors along with algo-
rithms for efficient recognition of human affective states
are now available and applicable for individuals with
ASD [18]. However, many improvements are needed to
attain significant success in treating individuals with aut-
ism, which depends on practical and clinical aspects. From
the practical perspective, many of the existing technologies
have limited capabilities in their performance and thus
limit the success in the therapeutic approach of children
with ASD. This is especially significant for wearable
hardware sensors that can provide feedback from the
individuals with ASD during the therapeutic session. More
studies must be performed to generate a reliable emotional,
attentional, behavioral or other type of feedback that is
essential to tailoring the special education methods to
better suit people with autism. Clinically, most of the ICT
proposals have not been validated outside the context of
proof of concept studies. More studies should be performed
to assess whether ICT architectures and devices are clini-
cally relevant. To overcome some of the limitations of ICT
proposals, social robotics have emerged in the field of
autism.
Social robotics should enable more natural and physical
interactions in terms of communication, emotion and social
abilities. However, some authors (e.g., [123]) have high-
lighted the anecdotal results of introducing robots into
experiments or therapeutic sessions with ASD individuals.
In particular, these researchers wondered why no one has
yet studied the best way to integrate robots into therapy
sessions. For this reason, they have remained very critical
of the results obtained in the field of robotics and ASD.
However, as an emerging field, there are several open
questions that must be addressed to improve the research
quality. What are the best roles for robots in therapy? How
could we best integrate robots into interventions? Addi-
tionally, among individuals with ASD, who is best suited
for this approach? These questions are some of the chal-
lenges future research will face. Taking into account the
recent advances in early developmental approaches, we
believe that focusing on two skills, such as imitation and
joint attention, will have an important clinical impact
because (1) they belong to the agenda of the intervention
program with the best evidence in young children with
ASD [38] and (2) both skills have already shown promis-
ing results in the field of SSP.
Moreover, in a recent study [20], we proposed a new
experimental paradigm that explored the question of how
the robot learning reacts to different participants (adults,
TD children and children with ASD). This new approach
allowed to analyze and to understand how cognitive
models (as conceptualized through cognitive computation)
are influenced by groups of participants. We investigated
posture learning through imitation between a human and a
robot. Our specific aim was to assess the influence of
participants on robot learning. First, the results showed that
the robot could learn the task autonomously by interacting
with groups of participants. The robot was able to learn,
recognize and imitate many specific postures autono-
mously through an imitation game. Robot learning was
based on a sensory motor architecture whereby neural
networks enabled the robot to associate what it did with
what it saw. In this study, metrics were used to evaluate the
behavior of different groups of participants interacting with
the robot. The metrics were used to assess the quality and
complexity of the interaction and to evaluate how the robot
reacted to different groups of participants. The results
showed that robot learning depended on the participants.
Here, the complexity was assessed in terms of the number
of neurons needed to learn. Learning this task has a ’’neural
cost’’ or a ’’cognitive cost’’ for the robot, i.e., the robot
needs more or less neurons. The results showed that more
neurons were recruited when the robot interacted with
children with ASD than when the robot interacted with TD
children or with adults.
The question of how to evaluate the interaction between
a human and a robot (or between a human and another
human) is crucial if we want to succeed in the challenges
described above. To do so, we propose addressing the
issues of interpersonal synchrony and multimodal integra-
tion during interactions because they appear to be key
issues in applying ICT in children with ASD. One way to
evaluate these interactions is to take into account the
dynamics of communication, such as synchrony, which
refers to individuals’ temporal coordination during social
interactions. Synchrony has received multidisciplinary
attention because of its role in early development, language
learning and SSP [42]. Synchrony appears to be a key
Cogn Comput
123
Author's personal copy

metric in human communication dynamics and interac-
tion [148]. Evaluating human/robot interaction means
analyzing, understanding and characterizing the commu-
nication between two partners. So far, few models have
been proposed to capture mimicry in dyadic interactions.
Mimicry is usually considered within the larger framework
of assessing interactional synchrony, which is the coordi-
nation of movement between individuals, with respect to
both the timing and form, during interpersonal communi-
cation [14]. The first step in computing synchrony is to
extract the relevant features of the dyad’s motion. Some
studies [5, 27, 147, 153] have focused on head motion,
which can convey emotion, acknowledgement or active
participation in an interaction. Other studies have captured
the global movements of the participants with motion
energy imaging [2, 121] or derivatives [41, 138]. Then, a
measure of similarity is applied between the two time
series. Several studies have also used a peak-picking
algorithm to estimate the time lag between partners [2, 5,
19]. Authors in [97] recently proposed an unsupervised
approach to measuring immediate synchronous and asyn-
chronous imitations between two partners. The proposed
model is based on the following two steps: detection of
interest points in images and evaluation of the similarity
between actions. The current challenges to mimicry
involve the characterization of both temporal coordination
(synchrony) and content coordination (behavior matching)
in a dyadic interaction [42].
Although a number of research issues need to be
solved, we believe that the state of the art of social
robotics should allow researchers, guided by multidisci-
plinary approaches, to develop new experimental settings
that can integrate interactions between children with ASD
and robots, with the aim of analyzing children’s behav-
iors. We believe that the robotic scenario is an excellent
way to elicit behaviors by interacting with the child and,
in return, analyzing the child’s behavior and adapting to
it. In such a case, introducing robots into therapy would
be of great clinical interest. From our view, creating
experimental protocols and databases that contribute to
the research of SSP for ASD, interdisciplinary approaches
and teams are required. By gathering researchers from
psychopathology, neuroscience, engineering and robotics,
we may efficiently address some of the aforementioned
challenges [30].
Acknowledgments This study was supported by a grant from the
European Commission (FP7: Michelangelo under Grant agreement n
288241) and the fund ‘‘Entreprendre pour aider.’’ The funding
agencies and the University were not involved in the study design,
collection, analysis and interpretation of data, writing of the paper or
the decision to submit the paper for publication. We would like to
thank MICHELANGELO Study Group (S. Bonfiglio, K. Maharatna,
E. Tamburini, A. Giuliano, M. Donnelly) for interesting discussions.
References
1. Aleotti J, Caselli S. Robust trajectory learning and approxima-
tion for robot programming by demonstration. Rob Auton Syst.
2006;54(5):409–13.
2. Altmann U. Studying movement synchrony using time series
and regression models. In: Esposito IA, Hoffmann R, Hu
¨
bler S,
Wrann B, editors. Program and Abstracts of the COST 2102.
International training school on cognitive behavioural systems.
Dresden, Germany, 21–25 February; 2011. p. 23.
3. Andry P, Gaussier P, Moga S, Banquet J, Nadel J. Learning and
communication in imitation: an autonomous robot perspective.
IEEE Trans Syst Man Cybernet Part A. 2001;31(5):431–44.
4. Andry P, Gaussier P, Nadel J, Hirsbrunner B. Learning invariant
sensory-motor behaviors: a developmental approach of imitation
mechanisms. Adapt Behav, 2004;12(2):117–40.
5. Ashenfelter K, Boker S, Waddell J, Vitanov N, Abadjieva E.
Spatiotemporal symmetry and multifractal structure of head
movements during dyadic conversation. J Exp Psychol Hum
Percept Perform. 2009;35(4):1072.
6. Bakker P, Kuniyoshi Y. Robot see, robot do: an overview of
robot imitation. In: AISB96 Workshop on Learning in Robots
and Animals. 1996; p. 3–11.
7. Baldwin J. Development and evolution. New York: The Mac-
millan company; 1902.
8. Baron-Cohen S. Mindblindness: an essay on autism and theory
of mind. Cambridge: MIT press; 1997.
9. Baron-Cohen S, Wheelwright S. ’Obsessions’ in children with
autism or asperger syndrome. Content analysis in terms of core
domains of cognition. Br J Psychiatry. 1999;175(5):484–90.
10. Bashshur R. Telemedicine and health care. Telemed J E Health.
2002;8(1):512.
11. Bauminger N, Gal E, Goren-Bar D. Enhancing social commu-
nication in high functioning children with autism through a co-
located interface. 6th International Workshop on Social Intelli-
gence Design, Trento 2007.
12. Bekele E, Zheng Z, Swanson A, Crittendon J, Warren Z, Sarkar
N. Understanding how adolescents with autism respond to facial
expressions in virtual reality environments. IEEE Trans Vis
Comput Graph. 2013;19(4):711–20.
13. Bellani M, Fornasari L, Chittaro L, Brambilla P. Virtual reality in
autism: state of the art. Epidemiol Psychiatr Sci. 2011;20(3):235–8.
14. Bernieri F, Reznick J, Rosenthal R. Synchrony, pseudosynchrony,
and dissynchrony:measuring the entrainment process in mother-
infant interactions. J Personal Soc Psychol. 1988;54(2):243.
15. Berthouze L, Bakker P, Kuniyoshi Y. Learning of oculo-motor
control: a prelude to robotic imitation. In: Intelligent Robots and
Systems’ 96, IROS 96, Proceedings of the 1996 IEEE/RSJ
International Conference on, vol. 1. 1996; p. 376–381. IEEE.
16. Billard A, Hayes G. Learning to communicate through imitation
in autonomous robots. In: Artificial Neural NetworksICANN’97.
Springer, Berlin; 1997; p. 763–768.
17. Bird G, Leighton J, Press C, Heyes C. Intact automatic imitation
of human and robot actions in autism spectrum disorders. Proc R
Soc B Biol Sci. 2007;274(1628):3027–31.
18. Blocher K, Picard R. Affective social quest: emotion recognition
therapy for autistic children. Socially Intelligent Agents: creat-
ing relationships with computers and robots; 2002. p. 133140 .
19. Boker S, Xu M, Rotondo J, King K. Windowed cross-correlation and
peak picking for the analysis of variability in the association between
behavioral time series. Psychol Methods. 2002;7(3):338–55.
20. Boucenna S, Anzalone S, Tilmont E, Cohen D, Chetouani M.
Learning of social signatures through imitation game between a
robot and a human partner. IEEE Trans Auton Ment Dev, in
revision; 2014a.
Cogn Comput
123
Author's personal copy

21. Boucenna S, Delaherche E, Chetouani M, Gaussier P. A new
approach for learning postures : an imitation game between a
human and a robot. In: IEEE/RSJ International Conference on
Robots and Systems, (IROS 2012). Workshop : Cognitive
Neuroscience, Robotics; 2012. p. 1–4.
22. Boucenna S, Gaussier P, Andry P, Hafemeister L. Imitation as a
communication tool for online facial expression learning and rec-
ognition. In: Intelligent Robots and Systems (IROS), 2010 IEEE/
RSJ International Conference on. IEEE. 2010; p. 5323–5328.
23. Boucenna S, Gaussier P, Andry P, Hafemeister L. A robot learns
the facial expressions recognition and face/non-face discrimi-
nation through an imitation game. Int J Soc Robot, to appear
2014b.
24. Boucenna S, Gaussier P, Hafemeister L. Development of joint
attention and social referencing. In: Development and Learning
(ICDL), 2011 IEEE International Conference on, vol 2. IEEE.
2011; p. 1–6.
25. Calinon S, Guenter F, Billard A. On learning, representing, and
generalizing a task in a humanoid robot. IEEE Trans Syst Man
Cybernet Part B Cybernet. 2007;37(2):286–98.
26. Campbell C, Peters R, Bodenheimer R, Bluethmann W, Huber
E, Ambrose R. Superpositioning of behaviors learned through
teleoperation. IEEE Trans Robot. 2006;22(1):79–91.
27. Campbell N. Multimodal processing of discourse information;
the effect of synchrony. In: Universal Communication, 2008.
ISUC’08. Second International Symposium on. IEEE, 2008;
p. 12–15.
28. Carpendale J, Lewis C. Constructing an understanding of the
mind: the development of children’s social understanding within
social interaction. Behav Brain Sci. 2004;27:79–151.
29. Carpenter M, Nagell K, Tomasello M. Social cognition, joint
attention, and communicative competence from 9 to 15 months
of age. Monogr Soc Res Child Dev. 1998;63(4):i–vi, 1–143.
30. Chaby L, Chetouani M, Plaza M, Cohen D. Exploring multi-
modal social-emotional behaviors in autism spectrum disorders.
In: Workshop on Wide Spectrum Social Signal Processing, 2012
ASE/IEEE International Conference on Social, Computing.
2012; p. 950–954.
31. Chaminade T, Da Fonseca D, Rosset D, Lutcher E, Cheng G,
Deruelle C. Fmri study of young adults with autism interacting
with a humanoid robot. In: RO-MAN, 2012 IEEE. IEEE, 2012;
p. 380–385.
32. Chetouani M, Mahdhaoui A, Ringeval F. Time-scale feature
extractions for emotional speech characterization. Cognit
Comput. 2009;1(2):194–201.
33. Cohen D. Traumatismes et traces : donne
´
s expe
´
rimentales.
Neuropsychiatr Enfance Adolesc. 2012;60:315–23.
34. Costa S, Santos C, Soares F, Ferreira M, Moreira F. Promoting
interaction amongst autistic adolescents using robots. In Engi-
neering in Medicine and Biology Society (EMBC), 2010 Annual
International Conference of the IEEE. IEEE; 2010; p. 3856–3859.
35. Damasio A. Descartes’ error. New York: Putnam’s Sons; 1994.
36. Dautenhahn K. Roles and functions of robots in human society:
implications from research in autism therapy. Robotica.
2003;21(4):443–52.
37. Dautenhahn K, Werry I. Towards interactive robots in autism
therapy: Background, motivation and challenges. Pragmat
Cognit. 2004;12(1):1–35.
38. Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J,
Donaldson A, Varley J. Randomized, controlled trial of an
intervention for toddlers with autism: the early start denver
model. Pediatrics. 2009;125(1):17–23.
39. de Rengerve
´
A, Boucenna S, Andry P, Gaussier P. Emergent
imitative behavior on a robotic arm based on visuo-motor
associative memories. In: Intelligent Robots and Systems
(IROS), 2010 IEEE/RSJ International Conference on. IEEE,
2010; p. 1754–1759.
40. Decety J, Jackson P. The functional architecture of human
empathy. Behav Cogn Neurosci Rev. 2004;3(2):71–100.
41. Delaherche E, Chetouani M. Multimodal coordination: explor-
ing relevant features and measures. In: Proceedings of the 2nd
international workshop on Social signal processing. ACM: 2010;
p. 47–52.
42. Delaherche E, Chetouani M, Mahdhaoui A, Saint-Georges C,
Viaux S, Cohen D. Interpersonal synchrony: a survey of eval-
uation methods across disciplines. IEEE Trans Affect Comput.
2012;3(3):349–65.
43. Diamond B, Shreve G, Bonilla J. Telerehabilitation, cognition
and user-accessibility. NeuroRehabilitation. 2003;18(2):1717.
44. Diehl J, Schmitt L, Villano M, Crowell C. The clinical use of
robots for individuals with autism spectrum disorders: a critical
review. Res Autism Spectr Disord. 2012;6(1):249–62.
45. Dillmann R. Teaching and learning of robot tasks via observa-
tion of human performance. Robot Auton Syst. 2004;47(2):
109–16.
46. DiSalvo D, Oswald D. Peer-mediated interventions to increase
the social interaction of children with autism: consideration of
peer experiences. Focus Autism Other Dev Disabl. 2002;17(4):
198–208.
47. Duquette A, Michaud F, Mercier H. Exploring the use of a
mobile robot as an imitation agent with children with low-
functioning autism. Auton Robots. 2008;24(2):147–57.
48. Eddon G. Danny’s rooms. In: proceedings of the john hopkins
national search for computing applications to assist persons with
disabilities. IEEE Computing Society press, 1992; p. 7879.
49. Emery N. The eyes have it: the neuroethology, function and evo-
lution of social gaze. Neurosci BiobehavRev. 2000;24(6):581–604.
50. Enyon A. Computer interaction: an update on the avatar pro-
gram. Communication, Summer. 1997; p. 18.
51. Esposito A. The perceptual and cognitive role of visual and
auditory channels in conveying emotional information. Cognit
Comput. 2009;1(3):268–78.
52. Fabri M. Emotionally expressive avatars for collaborative vir-
tual environments. A thesis submitted in partial fulfillment of the
requirements of Leeds Metropolitan University for the degree of
Doctor of Philosophy 2006.
53. Fabri M, Elzouki S, Moore D. Emotionally expressive avatars
for chatting, learning and therapeutic intervention, human–
computer interaction. HCI Intelligent Multimodal Interaction
Environments, 2007; p. 275–285.
54. Feil-Seifer D, Mataric
´
M. Toward socially assistive robotics for
augmenting interventions for children with autism spectrum disor-
ders. In: Experimental robotics. Springer, Berlin. 2009; p. 201–210.
55. Feil-Seifer D, Mataric M. Automated detection and classifica-
tion of positive vs. negative robot interactions with children with
autism using distance-based features. In: Human-Robot Inter-
action (HRI), 2011 6th ACM/IEEE International Conference on.
IEEE: 2011; p. 323–330.
56. Flores M, Musgrove K, Renner S, Hinton V, Strozier S, Franklin
S, Hil D. A comparison of communication using the apple ipad
and a picture-based system. Augment Altern Commun.
2012;28(2):74–84.
57. Fong T, Nourbakhsh I, Dautenhahn K. A survey of socially
interactive robots. Robo Auton Syst. 2003;42(3):143–66.
58. Franc¸ois D, Powell S, Dautenhahn K. A long-term study of chil-
dren with autism playing with a robotic pet: taking inspirations
from non-directive play therapy to encourage children’s proac-
tivity and initiative-taking. Interact Stud. 2009;10(3):324–73.
59. Frith U, Morton J, Leslie A. The cognitive basis of a biological
disorder: autism. Trends Neurosc. 1991;14(10):434438.
Cogn Comput
123
Author's personal copy

60. Fujimoto I, Matsumoto T, De Silva P, Kobayashi M, Higashi M.
Mimicking and evaluating human motion to improve the imi-
tation skill of children with autism through a robot. Int J Soc
Robot. 2011;3(4):349–57.
61. Gal E, Goren-Bar D, Gazit E, Bauminger N, Cappelletti A,
Pianesi F, Stock O, Zancanaro M, Weiss PL. Enhancing social
communication through story-telling among high-functioning
children with autism. Intelligent Technologies for Interactive
Entertainment, 2005; p. 320–323.
62. Golan O, Ashwin E, Granader Y, McClintock S, Day K, Leggett
V, Baron-Cohen S. Enhancing emotion recognition in children
with autism spectrum conditions: an intervention using animated
vehicles with real emotional faces. J Autism Dev Disord.
2010;40(3):269–79.
63. Gonzalez J, Cabrera M, Gutierrez F. Using videogames in
special education. Comput Aid Syst Theory, EUROCAST.
2007; p. 360–7.
64. Green, S. (1993). Computer-based simulations in the education
and assessment of autistic children. in: Rethinking the Roles of
Technology in Education, Tenth International Conference on
Technology and Education, Massachusetts Institute of Tech-
nology, Cambridge, MA, 1:334336.
65. Guenter F, Hersch M, Calinon S, Billard A. Reinforcement
learning for imitating constrained reaching movements. Adv
Robot. 2007;21(13):1521–44.
66. Hayes G, Demiris J. A robot controller using learning by imi-
tation. Department of Artificial Intelligence, University of
Edinburgh; 1994.
67. Herrera G, Alcantud F, Jordan R, Blanquer A, Labajo A, De
pablo C. Development of symbolic play through the use of
virtual reality tools in children with autistic spectrum disorders.
Autism. 2008;12:143157.
68. Hersch M, Guenter F, Calinon S, Billard A. Dynamical system
modulation for robot learning via kinesthetic demonstrations.
IEEE Trans Robot. 2008;24(6):1463–7.
69. Hetzroni O, Juman T. Effects of a computer-based intervention
program on the communicative functions of children with aut-
ism. J Autism Dev Disord. 2011;34(2):95–113.
70. Hileman C. Computer technology with autistic children. In:
Autism Society of America National Conference. Milwaukee,
Wisconsin; 1996.
71. Hopkins I, Gower M, Perez T, Smith D, Amthor F, Wimsatt F,
Biasini F. Avatar assistant: improving social skills in students
with an asd through a computer-based intervention. J Autism
Dev Disord. 2011;41(11):1543–55.
72. Horace H, Belton K. Smart ambience games for children with
learning difficulties. Technol E Learn Digit Entertain, 2006;
484–493.
73. Howlin P, Baron-Cohen S, Hadwin J. Teaching children with
autism to mind-read: a practical guide for teachers and parents.
New York: Wiley; 1999.
74. Hutinger P, Rippey R. How five preschool children with autism
responded to computers. 1997.
75. Ijspeert A, Nakanishi J, Schaal S. Movement imitation with
nonlinear dynamical systems in humanoid robots. In: Robotics
and Automation, 2002. Proceedings. ICRA’02. IEEE Interna-
tional Conference on, vol 2. IEEE; 2002; pages 1398–1403.
76. Jones S. Imitation in infancy the development of mimicry.
Psychol Sci. 2007;18(7):593–9.
77. Jones S. The development of imitation in infancy. Philos Trans
R Soc B Biol Sci. 2009;364(1528):2325.
78. Jordan R. Computer assisted education for individuals with
autism. Paper presented at the Autisme France 3rd International
Conference 1995.
79. Jordan R. Multidisciplinary work for children with autism. Educ
Child Psychol. 2001;18(2):5–14.
80. Jowett E, Moore D, Anderson A. Using an ipad-based video
modelling package to teach numeracy skills to a child with an
autism spectrum disorder. Dev Neurorehabil. 2012;15(4).
81. Kagohara D, van der Meer L, Ramdoss S, O’Reilly M, Lancioni
G, Davis T, Rispoli M, Lang R, Marschik P, Sutherland D,
Green V, Sigafoos J. Using ipods and ipads in teaching pro-
grams for individuals with developmental disabilities: a sys-
tematic review. Res Dev Disabil. 2013;34(1):147–56.
82. Kaliouby R, Picard R, Barron-Cohen S. Affective computing
and autism. Ann N Y Acad Sci, 2006; P. 228248.
83. Kaplan F, Hafner V. The challenges of joint attention. Interact
Stud. 2006;7(2):135–69.
84. Konstantinidis E, Bamidis P, Koufogiannis D. Development of a
generic and flexible human body wireless sensor network. In
Proceedings of the 6th European Symposium on Biomedical
Engineering (ESBME) 2008.
85. Konstantinidis E, Luneski A, Frantzidis C, Pappas C, Bamidis P.
A proposed framework of an interactive semi-virtual environ-
ment for enhanced education of children with autism spectrum
disorders. The 22nd IEEE International Symposium on Com-
puter-Based Medical Systems (CBMS) 2009.
86. Kozima H, Michalowski M, Nakagawa C. Keepon: a playful
robot for research, therapy, and entertainment. Int J Soc Robot.
2009;1(1):3–18.
87. Kozima H, Nakagawa C, Yasuda Y. Children-robot interaction:
a pilot study in autism therapy. Prog Brain Res. 2007;164:385.
88. Kuniyoshi Y. The science of imitation-towards physically and
socially grounded intelligence. In: Special Issue TR-94001, Real
World Computing Project Joint Symposium, Tsukuba-shi, Iba-
raki-ken. Citeseer; 1994.
89. La
´
nyi CS, Tilinger A. Multimedia and virtual reality in the
rehabilitation of autistic children. In: Computers helping people
with special needs. Heidelberg: Springer; 2004. p. 22–8.
90. Liu C, Conn K, Sarkar N, Stone W. Physiology-based affect
recognition for computer-assisted intervention of children with
autism spectrum disorder. Int J Hum Comput Stud. 2008;
66(9):662–77.
91. Lum P, Uswatte G, Taub E. A telerehabilitation approach to
delivery of constraint-induced movement therapy. J Rehabil R
D. 2006;43(3):391400.
92. Luneski, A., Konstantinidis, E., Hitoglou-Antoniadou, M., and
Bamidis, P. (2008). Affective computer-aided learning for
autistic children. 1st Workshop of Child, Computer and Inter-
action (WOCCI 08). Chania, Greece.
93. Marin-Urias L, Sisbot E, Alami R. Geometric tools for per-
spective taking for human-robot interaction. In: Artificial
Intelligence, 2008. MICAI’08. Seventh Mexican International
Conference on. IEEE; 2008; p. 243–249.
94. Marin-Urias L, Sisbot E, Pandey A, Tadakuma R, Alami R.
Towards shared attention through geometric reasoning for
human robot interaction. In: Humanoid Robots, 2009. Human-
oids 2009. 9th IEEE-RAS International Conference on. IEEE;
2009; p. 331–336.
95. Marnik J, Szela M. Multimedia program for teaching autistic
children. Inf Technol Biomed ASC. 2008;47:505512.
96. Maurer A, Hersch M, Billard A. Extended hopfield network for
sequence learning: Application to gesture recognition. In: Arti-
ficial Neural Networks: Biological Inspirations-ICANN 2005.
Springer, Berlin; 2005; p. 493–498.
97. Michelet S, Karp K, Delaherche E, Achard C, Chetouani M.
Automatic imitation assessment in interaction. In: Human
Behavior Understanding (IROS’12 workshop)., page To appear
2012.
98. Mitchell P, Parsons S, Leonard A. Using virtual environments
for teaching social understanding to 6 adolescents with autistic
spectrum disorders. J Autism Dev Disord. 2006;3(37):589–600.
Cogn Comput
123
Author's personal copy

99. Mitchell P, Parsons S, Leonard A. Using virtual environments
for teaching social understanding to 6 adolescents with autistic
spectrum disorders. J Autism Dev Disord. 2007;37:589600.
100. Murdock L, Ganz J, Crittendon J. Use of an ipad play story to
increase play dialogue of preschoolers with autism spectrum
disorders. J Autism Dev Disord. 2013;43(9):2174–89.
101. Murray D. Autism and information technology: therapy with
computers. Autism Learn Guide Good Pract, 1997; p. 100117.
102. Nadel J, Simon M, Canet P, Soussignan R, Blancard P, Cana-
mero L, Gaussier P. Human responses to an expressive robot. In:
Procs of the Sixth International Workshop on Epigenetic
Robotics. Lund University; 2006.
103. Nagai Y, Hosoda K, Morita A, Asada M. A constructive model
for the development of joint attention. Connect Sci. 2003;
15(4):211–29.
104. Narzisi A, Colombi C, Fusar P, Balottin U, Muratori F. Non-
pharmacological treatments in autism spectrum disorders: an
overview on early interventions for pre-schoolers. Curr Clin
Pharmacol. 2014;9(1):1–10.
105. Narzisi A, Muratori F, Calderoni S, Fabbro F, Urgesi C. Neu-
ropsychological profile in high functioning autism spectrum
disorders. J Autism Dev Dis. 2013;43(8):1895–909.
106. Orvalho V, Miranda J, Sousa A. Facial synthesys of 3d avatars
for therapeutic applications. Stud Health Technol Inform. 2009;
144:96–8.
107. Ospina M, Seida J, Clark B, Karkhaneh M, Hartling L, Tjosvold
L, Vandermeer B, Smith V. Behavioural and developmental
interventions for autism spectrum disorder: a clinical systematic
review. PLoS One, 2008;3(11).
108. Papert S. Mindstorms: Children, computers, and powerful ideas.
New York: Basic Books Inc; 1980.
109. Pardowitz M, Dillmann R. Towards life-long learning in
household robots: The piagetian approach. In: Development and
Learning, 2007. ICDL 2007. IEEE 6th International Conference
on. IEEE; 2007; p. 88–93.
110. Pardowitz M, Knoop S, Dillmann R, Zollner R. Incremental
learning of tasks from user demonstrations, past experiences,
and vocal comments. IEEE Trans Syst Man Cybernet Part B
Cybernet. 2007;37(2):322–32.
111. Parmanto B, Saptono A, Ferrydiansyah R, Sugiantara I. Trans-
coding biomedical information resources for mobile handhelds.
Proceedings of the 38th Annual Hawaii International Confer-
ence on System Sciences (HICSS’05). IEEE Computer Society,
2005;6:151–158.
112. Parsons S, Beardon L, Neale H, Reynard G, Eastgate R, Wilson
J, Cobb S, Benford S, Mitchell P, Hopkins E. Development of
social skills amongst adults with asperger’s syndrome using
virtual environments: the ’as interactive’ project. In: Sharkey P,
Cesarani A, Pugnetti L, Rizzo A, editors. 3rd ICDVRAT.
Sardinia Italy: University of Reading; 2000.
113. Parsons S, Leonard A, Mitchell P. Virtual environments for
social skills training: comments from two adolescents with
autistic spectrum disorder. Comput Educ. 2006;47(2):186–206.
114. Piaget J. La naissance de l’intelligence chez l’enfant, vol. 968.
Paris: Delachaux et Niestle
´
; 1977.
115. Picard R. Emotion research by the people, for the people. Emot
Rev. 2010;2(3):250–4.
116. Pierno A, Mari M, Lusher D, Castiello U. Robotic movement
elicits visuomotor priming in children with autism. Neuro-
psychologia. 2008;46(2):448–54.
117. Pioggia G, Igliozzi R, Ferro M, Ahluwalia A, Muratori F, De
Rossi D. An android for enhancing social skills and emotion
recognition in people with autism. IEEE Trans Neural Syst
Rehabil Eng. 2005;13(4):507–15.
118. Pioggia G, Igliozzi R, Sica M, Ferro M, Muratori F, Ahluwalia
A, De Rossi D. Exploring emotional and imitational android-
based interactions in autistic spectrum disorders. J Cyber
Therapy Rehabil. 2008;1(1):49–61.
119. Premack D, Woodruff G. Does the chimpanzee have a theory of
mind? Behav Brain Sci. 1978;1(04):515–26.
120. Rajendran G, Mitchell P. Computer mediated interaction in as-
perger’s syndrome: the bubble dialogue program. Comput Educ.
2000;35:187–207.
121. Ramseyer F, Tschacher W. Nonverbal synchrony in psychother-
apy: coordinated body-movement reflects relationship quality and
outcome. J Consult Clin Psychol. 2011;79(3):284–95.
122. Ravindra P, De Silva S, Tadano K, Saito A, Lambacher S,
Higashi M. Therapeutic-assisted robot for children with autism.
In: Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ
International Conference on. IEEE; 2009; p. 3561–3567.
123. Ricks D, Colton M. Trends and considerations in robot-assisted
autism therapy. In: Robotics and Automation (ICRA), 2010
IEEE International Conference on. IEEE; 2010; p. 4354–4359.
124. Robins B, Dautenhahn K, Dubowski J. Does appearance matter
in the interaction of children with autism with a humanoid
robot? Interact Stud. 2006;7(3):509–42.
125. Robins B, Dautenhahn K, Te Boekhorst R, Billard A. Robotic
assistants in therapy and education of children with autism: can
a small humanoid robot help encourage social interaction skills?
Univers Access Inf Soc. 2005;4(2):105–20.
126. Rosset D, Rondan C, Fonseca D, Santos A, Assouline B,
Deruelle C. Typical emotion processing for cartoon but not for
real faces in children with autistic spectrum disorders. J Autism
Dev Disord. 2008;38(5):919–25.
127. Sadeghipour A, Kopp S. Embodied gesture processing: motor-
based integration of perception and action in social artificial
agents. Cognit Comput. 2011;3(3):419–35.
128. Scassellati B. Imitation and mechanisms of joint attention: a
developmental structure for building social skills on a humanoid
robot. In: Computation for metaphors, analogy, and agents.
Springer, Berlin; 1999; p. 176–195.
129. Scassellati B. Foundations for a Theory of Mind for a Humanoid
Robot. PhD thesis, Massachusetts Institute of Technology; 2001.
130. Scassellati B. How social robots will help us to diagnose, treat,
and understand autism. In: Robotics research. Springer, Berlin;
2007; p. 552–563.
131. Scassellati B, Admoni H, Mataric M. Robots for use in autism
research. Ann Rev Biomed Eng. 2012;14:275–94.
132. Serret S. Jestimule, a serious game for autism spectrum disor-
ders. Neuropsychiatr Enfance Adolesc. 2012;60(5):59.
133. Simon M, Canet P, Soussignan R. L’enfant face a
`
des expres-
sions robotiques et humaines. Enfance. 2007;59(1):59–70.
134. Sisbot E, Marin L, Alami R. Spatial reasoning for human robot
interaction. In: Intelligent Robots and Systems, 2007. IROS
2007. IEEE/RSJ International Conference on. IEEE; 2007;
p. 2281–2287.
135. Steil J, Ro
¨
thling F, Haschke R, Ritter H. Situated robot learning
for multi-modal instruction and imitation of grasping. Robot
Auton Syst. 2004;47(2):129–41.
136. Stribling P, Rae J, Dickerson P. Using conversation analysis to
explore the recurrence of a topic in the talk of a boy with an
autism spectrum disorder. Clin Linguist Phon. 2009;23(8):
555–82.
137. Strickland D. Brief report: two case studies using virtual reality
as a learning tool for autistic children. J Autism Dev Disord.
1996;26(6):651–9.
138. Sun X, Nijholt A. Multimodal embodied mimicry in interaction.
In: Analysis of Verbal and Nonverbal Communication and
Enactment. The Processing Issues. Springer; 2011; p. 147–153.
139. Takacs B. Special education and rehabilitation: teaching and
healing with interactive graphics. IEEE Comput Graph Appl.
2005;25(5):40–8.
Cogn Comput
123
Author's personal copy

140. Tapus A, Mataric M, Scassellati B. Socially assistive robotics.
IEEE Robot Autom Mag. 2007;14(1):35.
141. Tartaro A, Cassell J. Authorable virtual peers for autism spec-
trum disorders, conference on human factors in computing
systems. CHI ’07, San Jose, CA, USA, 2007;p. 1677–1680.
142. Thill S, Pop C, Belpaeme T, Ziemke T, Vanderborght B. Robot-
assisted therapy for autism spectrum disorders with (partially)
autonomous control: challenges and outlook. Paladyn. 2012;3(4):
209–17.
143. Thorndike E. Animal intelligence: An experimental study of the
associative processes in animals. Psychol Monogr General Appl,
1898;2(4):i–109.
144. Tomasello M. Joint attention as social cognition. Joint attention:
its origins and role in development; 1995. p. 103–130.
145. Tomasello M, Farrar M. Joint attention and early language.
Child Dev, 1986; p. 1454–1463.
146. Trepagnier C, Michael R. Telerehabilitation and virtual reality
technology for rehabilitation: preliminary results. Proceedings
of the 1999 International Technology and Persons with Dis-
abilities Conference, 1999.
147. Varni G, Volpe G, Camurri A. A system for real-time multi-
modal analysis of nonverbal affective social interaction in user-
centric media. IEEE Trans Multimed. 2010;12(6):576–90.
148. Vinciarelli A, Pantic M, Bourlard H. Social signal processing:
survey of an emerging domain. Image Vis Comput. 2009;
27(12):1743–59.
149. Vismara L, Rogers S. Efficacy of the early start denver model
parent intervention for toddlers with asd delivered via internet
technology. Philadelphia (USA): Oral presentation at Interna-
tional Meeting for Autism Research; 2010.
150. Wainer J, Ferrari E, Dautenhahn K, Robins B. The effectiveness
of using a robotics class to foster collaboration among groups of
children with autism in an exploratory study. Personal Ubiqui-
tous Comput. 2010;14(5):445–55.
151. Wallon H. De l’acte a
`
la pense
´
e 1942.
152. Wang M, Reid D. Virtual reality in pediatric neurorehabilitation:
attention deficit hyperactivity disorder. Autism and cerebral
palsy. Neuroepidemiology. 2011;36:218.
153. Weissman O, Delaherche E, Rondeau M, Chetouani M, Cohen
D, Feldman R. Oxytocin shapes parental motion characteristics
during parent-infant interaction. Biol Lett, page in revision.
154. Welch K, Lahiri U, Warren Z, Sarkar N. An approach to the
design of socially acceptable robots for children with autism
spectrum disorders. Int J Soc Robot. 2010;2(4):391–403.
155. Whiten A, Ham R. On the nature and evolution of imitation in
the animal kingdom: reappraisal of a century of research. Adv
Study Behav. 1992;21:239–83.
156. Winters J. Telerehabilitation research: emerging opportunities.
Annu Rev Biomed Eng. 2002;4:287320.
157. Yucel Z, Salah A, Merigli C, Meric¸li T. Joint visual attention
modeling for naturally interacting robotic agents. In: Computer
and Information Sciences, 2009. ISCIS 2009. 24th International
Symposium on. IEEE; 2009; pages 242–247.
Cogn Comput
123
Author's personal copy