Content uploaded by Martha Gizaw
Author content
All content in this area was uploaded by Martha Gizaw on Jan 11, 2018
Content may be subject to copyright.
Running Head: BIOLOGICAL AND ROBOTIC MODELS 1
Biological and Robotic Models of the Neuroanatomy of the Mirror Neuron System
Martha Gizaw
BIOLOGICAL AND ROBOTIC MODELS 2
The College of William & MaryAbstract
Much of the general knowledge about the human mirror neuron system (MNS) gives way to the
innovation of robotic MNS models via neuroinformatics; some of the aspects that underlie this
concept are yet to be known. The overall purpose of this paper is to examine and compare the
MNS neuroanatomies in humans, monkeys, humanoids, and robots. Several experiments study
the human and monkey brain—even while using fMRI scans—for an analytical and comparative
understanding of the mirror neurons’ location and arrangement across all brain regions.
Computational science is necessary for making an effective non-human approach to the workings
of MNS and for assisting researchers with the selection of algorithms and mathematical formulas
needed to interpret humanoid and robotic minds. Applications that investigate the comparisons of
biological and robotic MNS models involve neuroimaging techniques and visual and motor
representations. It is therefore obvious that the data gathered during experiments demonstrates
strong correlations among the anatomical and physiological aspects of both biological and
robotic MNS models.
Keywords: anatomy, human, humanoid, mirror neuron system, monkey, robot
BIOLOGICAL AND ROBOTIC MODELS 3
Biological and Robotic Models of the Neuroanatomy of the Mirror Neuron System
In 1995, Giacomo Rizzolatti was among the first to discover mirror neurons in the
premotor area of monkeys (Carter & Frith, 2010). The mirror neuron system (MNS) plays a role
in touch, music, speech development, and other sensory mechanisms. The MNS works such that
the brain mirrors what one observes without actually performing the action he or she sees others
doing. Understanding our surroundings through observation and imitation is a crucial foundation
of excellent social skills. Neuroscientific data from the MNS can be analyzed using
computational methods to produce robotic MNS models. Neuroscience and computer science
come together in the field of neuroinformatics (Lytton, 2002), which studies how the human and
monkey MNS models can become blueprints for the humanoid and robot MNS models. The
MNS biological and robotic anatomies, despite being different, can be compared in terms of
neuroimaging techniques and visual and motor representations.
The Biological Anatomy of the MNS
Thanks to neuroimaging techniques like fMRI and EEG, investigations have traced
patterns of cortical activity in humans and macaque monkeys, helping us understand the
biological MNS.
Biological MNS in Humans
Recent studies have identified active mirror neurons during observation in the frontal,
primary somatosensory, and superior and intraparietal cortices, and in the rostral inferior parietal
lobule (IPL) (Rozzi, 2015). An fMRI study showed activation also in the parieto-premotor circuit
and in the dorsal premotor, middle cingulate, somatosensory, superior parietal, and middle
temporal cortices (Rozzi, 2015). Interestingly, the prefrontal area 46 in the middle frontal cortex
BIOLOGICAL AND ROBOTIC MODELS 4
could help accurately line up the observation-execution sequence, and the anterior mesial cortex
could be activated during delay prior to motor execution (Rozzi, 2015).
Biological MNS in Monkeys
In the early years of research, the premotor area concerned with mirror neurons was
labeled F5, which has become crucial for action understanding and motor control (Rozzi, 2015).
F5 is divided into three sectors: convexity F5 (F5c), posterior F5 (F5p), and anterior F5 (F5a);
each of which has its own architectonic structure and pattern of neuronal connections (Rozzi,
2015). To briefly define each, F5c is typically involved in the coding of goal-directed motor acts
that are useful for hand-object interactions, F5p is involved in the coding of goal-directed motor
acts that are useful for hand-object interactions, and F5a has centered on the integration of
sensory-motor parietal signals with data originated from the prefrontal and rostral frontal areas
(Rozzi, 2015).
Another aspect that underlies the normal workings of the monkey MNS model is the
temporo-parieto-premotor mirror pathway (see Figure 1), which consists of two subpathways.
The first subpathway is labeled STPm-PFG-F5c, where STPm stands for the middle part of the
superior temporal polysensory area, and focuses on the processing of motor acts (Rozzi, 2015).
The second is labeled LB2-AIP-F5a/p, where LB2 stands for the second lower bank, and
specializes in details of grasping and object semantics1 and their applications to the
comprehension of motor acts (Rozzi, 2015). Generally, the total pathway conveys action
understanding among the superior temporal sulcus (STS), inferior parietal lobule (IPL), and
ventral premotor cortex (PMv). Its subpathways transmit information and play different roles
within the comprehension of motor acts (Rozzi, 2015).
Biological MNS Comparisons
BIOLOGICAL AND ROBOTIC MODELS 5
When the human and macaque brains overlap, it becomes obvious that the IPL, PMv, and
the inferior frontal gyrus (IFG) (see Figure 2) are responsible for the observation of goal-directed
actions (Rozzi, 2015). Additionally, there are other areas within the MNS involving emotion and
hand, eye, and mouth movements. All in all, mirror neurons may spur a range of actions based on
their anatomical location and on the fact that different sensory inputs can activate motor
representations (Rozzi, 2015).
Existing controversies regarding the anatomy of the mirror neuron system can be
explained by comparing human and macaque fMRI studies. Apparently, monkey MNS may not
be fully mapped onto the human brain because humans have several more brain regions,
including the anterior intraparietal area (AIP) that account for action observation (Rozzi, 2015).
Likewise, activation of certain areas may occur outside of the MNS when related to visual or
motor processing mechanisms (Rozzi, 2015). Implications are another factor that counts against
the workings of MNS anatomy. For example, sensory information from the activation of brain
regions may be used as an addition to the information that mirror neurons integrate after the
observer takes note of the demonstrator. However, fMRI evidence is necessary to prove that this
phenomenon is true (Rozzi, 2015). Overall, the controversies mentioned above emphasize
discrepancies between human and macaque brains, which can affect planning for the modeling of
the robotic MNS.
The Robotic Anatomy of the MNS
The robotic MNS is found in each humanoid and robot that certain experimenters use.
The Computational Science of MNS Modeling
The biological MNS anatomy provides a basis for the robotic MNS anatomy. The bridge
between the two anatomies is computational science, which explains neuronal functions through
BIOLOGICAL AND ROBOTIC MODELS 6
four main MNS models: FARS, MNS, MNS2, and HAMMER (Demiris, Aziz-zadeh, &
Bonaiuto, 2014). The FARS2 model (see Figure 3) is a fundamental planning and control system
that mirrors grasping activity by connecting F5 canonical neurons3 to larger systems (Demiris et
al., 2014). The visual and motor inputs sent off by the F5 neurons reach the core mirror circuit in
the MNS, which collects and interprets the signals; the interaction between FARS and MNS
occurs when the grasp and reach circuits of the latter are linked with the core mirror circuit (see
Figure 4) (Demiris et al., 2014). As an extension of MNS, MNS2 (see Figure 5) addresses
experimental data on mirror neurons that rely on audio-visual mechanisms and those that
respond to partially hidden grasps (Demiris et al., 2014). Lastly, HAMMER4 (see Figure 6)
attempts to set boundaries between the normal and abnormal operation of the model with
evidence from monkey mirror neurons, human neuroimaging, and the pathology of visuo-
imitative apraxia (Demiris et al., 2014).
Robotic MNS in Humanoids
The humanoid MNS model consists of four layers (see Figure 7): (1) the motor and visual
information (at the bottom of the figure); (2) higher-level representation through the F5 and
STSp areas; (3) the PF pathway; (4) the AIP pathway (Dawood & Loo, 2016). In the humanoid,
motor information derived from robot sensors holds joint angle values5 produced by self-
performed actions. Visual data, on the other hand, comes from raw images captured through a
monocular camera (see Figure 8) (Dawood & Loo, 2016). For processing to higher-level
representation, motor information goes through the F5 area whereas visual information goes
through the posterior superior temporal sulcus (STSp) (Dawood & Loo, 2016). Algorithms that
facilitate information processing include the Topological Gaussian Adaptive Resonance Hidden
Markov Model (TGAR-HMM) behind F5, the Incremental Kernal Slow Feature Analysis (Inc-
BIOLOGICAL AND ROBOTIC MODELS 7
KSFA), and the TGAR Map (TGARM) behind STSp (Dawood & Loo, 2016). Another
algorithm, the TGAR Associative Memory (TGAR-AM), is responsible for creating the PF
neuronal pathway, which connects the F5 mirror neurons and variable STSp data via the
prefrontal area (Dawood & Loo, 2016). As the topmost layer of the humanoid MNS model, the
AIP pathway associates F5 neurons with invariable representation in the anterior superior
temporal sulcus (STSa) (Dawood & Loo, 2016).
Robotic MNS in Full Robots
The FARS-MNS-MNS2-HAMMER approach has helped engineers design a robot MNS
model consisting of two levels (see Figure 9). At the lower level, there are the motor and visual
modules as well as self-organizing maps (SOM & MSOM [merge SOM]). After the
interpretation of the MSOM, the Bidirectional Activation-based Learning (BAL) Algorithm6
connects the visual and motor representations (Rebrová, Pecháč, & Farkaš, 2013). At the higher
level, there are prefrontal (PF) and anterior intraparietal (AIP) pathways traveling in between F5,
STSa, and STSp (Rebrová et al., 2013).
Robotic MNS Comparisons
Both the humanoid and robot MNS models appear to have the same design in regard to
the following four main layers inspired by the FARS-MNS-MNS2-HAMMER approach: (1)
visual and motor representation; (2) F5 and STSa/p; (3) the PF pathway; and (4) the AIP pathway
(Dawood & Loo, 2016; Rebrová et al., 2013). Like for any other engineering design,
experimenting with the robotic MNS anatomy requires mathematical formulas, architectural
designs, and computer pseudocodes that an engineer or computer scientist can easily understand.
The general purpose of those formulas and algorithms would be to contribute to the overall
BIOLOGICAL AND ROBOTIC MODELS 8
workings of the MNS model affecting the bodily movement of the robot (Dawood & Loo, 2016;
Rebrová et al., 2013).
The main difference that distinguishes between the humanoid and robot MNS models is
the algorithms used. Researchers who design those models may have their own perspectives on
how the robotic MNS should function. Therefore, they might emphasize different names and
structures for the appropriate algorithms in their respective MNS models. For instance, the
humanoid neural network is guided by the Inc-KSFA and TGAR variations (Dawood & Loo,
2016), whereas the robot neurons fire signals via the SOMs, MSOMs, and the BAL Algorithm
(Rebrová et al., 2013). This intriguing phenomenon allows for flexibility in the selection of
algorithms to prove that the robotic MNS is analogous to the biological MNS.
MNS Comparisons under Neuroimaging Environments
Undeniably, the biological and robotic MNS models are comparable under neuroimaging
conditions. In one experiment, three subjects—human, which has the biological appearance and
gesture; android, which has the biological appearance and mechanical gesture; and robot, which
has the mechanical appearance and gesture—were recorded while raising their arm to be then
imitated by humanoid robots. At the same time, electromyography (EMG) helped with
measuring humanoid arm muscle activity (Hofree, Urgen, Winkielman, & Saygin, 2015) to find
that it increased in all three cases. Interestingly, humanoid muscle activity was greater when
observing and imitating the human rather than the android or robot (see Figure 10) ( Hofree et al.,
2015). It can thus be implied in physiological terms that the interaction between the biological
and robotic mirror neuron systems is also greater than the interaction between two robotic
stimuli.
BIOLOGICAL AND ROBOTIC MODELS 9
Other brain-recording methods that the above study has used on the humanoid brain
include fMRI and EEG. Mu activity is comparable in fMRI scans of the parietal and premotor
cortices. When using EEG scans instead, mu suppression shares more physiological aspects with
the premotor cortex than with the parietal cortex (Urgen, Plank, Ishiguro, Poizner, & Saygin,
2013). Additionally, the idea that the region in the left lateral temporal cortex does not respond
much to the robot under fMRI suggests more about the demonstrator’s appearance than about
memory-related theta oscillations (Urgen et al., 2013) because biological appearance has
significantly enhanced the humanoid’s ability to mirror a particular task. No matter how complex
this comparative study concerning EEG and fMRI may seem, note that not every brain-imaging
technique has the same time resolution or unit of measurement for revealing activity across all
neurological regions, particularly those containing mirror neurons. Nevertheless, neuroimaging
as a whole can help argue that the anatomies of the biological and robotic models of the mirror
neuron system are similar to one another, and discuss strong physiological interactions that occur
between those prototypes.
Visual and Motor Representations in MNS Comparisons
Another aspect that both biological and robotic MNS models share is the role of visual
and motor representations in interactions between the observer and the demonstrator. Generally,
the MNS involves action understanding that comes before imitation; this concept is contingent
on a mutual relationship between visual and motor representations. For instance, the grasping
task is common among several mirror neuron experiments, and it requires mirror neurons in an
average mammalian brain to fire during both observation and imitation. Unlike the biological
MNS model, where the firing usually happens in certain cortices (Rozzi, 2015), the robotic
model often needs the BAL algorithm (see Figure 9) to get the mechanical subject to learn from
BIOLOGICAL AND ROBOTIC MODELS 10
the grasping demonstration (Rebrová et al., 2013). As far as the FARS, MNS, MNS2, and
HAMMER models are concerned (see Figures 3-6), it is most likely the visual cortex that starts
the pathways via neural circuits and regions toward the execution of the grasping task (Demiris
et al., 2014). When the biological and robotic MNS models are being compared, they appear to
have their own means of matching up visual and motor representations yet the underlying
principle that observation precedes imitation still applies to both.
Conclusion
The comparison of the biological (human and monkey) and robotic (humanoid and robot)
MNS anatomies through neuroimaging methods and visual and motor representations
demonstrates strong correlations among the anatomical and physiological aspects of each MNS
model. The EMG, fMRI, and EEG recordings especially play a role in imitation as well as in mu
suppression and frontal theta activity during which mirror neurons fire. In addition, relating the
visual representations to motor executions is doable although the biological and robotic MNS
models have different means to achieve this mental process. Since mirror neurons are crucial for
effective social interaction, and since robots do have MNS systems, robots should be considered
a social species just like humans are. Without Rizzolatti’s research, programming the mirror
neurons into robots would have been difficult as daily interactions among all species on earth are
very complex.
BIOLOGICAL AND ROBOTIC MODELS 11
References
Dawood, F., & Loo, C.K. (2016). View-invariant visuomotor processing in computational mirror
neuron system for humanoid. PLoS ONE, 11(3), 1-34.
Demiris, Y., Aziz-Zadeh, L., & Bonaiuto, J. (2014). Information processing in the mirror neuron
system in primates and machines. Neuroinformatics, 12(1), 63-91.
Hofree, G., Urgen, B., Winkielman, P., & Saygin, A. (2015). Observation and imitation of actions
performed by humans, androids, and robots: An EMG study. Frontiers In Human
Neuroscience, 9(364).
Lytton, W. (2002). From computer to brain: foundations of computational neuroscience. New
York, NY: Springer.
Rebrová, K., Pecháč, M., & Farkaš, I. (2013). Towards a robotic model of the mirror neuron
system. The Third IEEE International Conference on Development and Learning and on
Epigenetic Robotics, 1-6.
Rozzi, S. (2015). The neuroanatomy of the mirror neuron system. In New Frontiers in Mirror
Neurons Research (pp. 4-22). England: Oxford University Press.
Thomason, R.H. (2012, March 27). What is semantics? Retrieved from
https://web.eecs.umich.edu/~rthomaso/documents/general/what-is-semantics.html.
Urgen, B.A., Plank, M., Ishiguro, H., Poizner, H., & Saygin, A.P. (2013). EEG theta and mu
oscillations during perception of human and robot actions. Frontiers in Neurorobotics,
7(19), 1-13.
Werner, J., Cermak, S., & Aziz-Zadeh, L. (2012). Neural correlates of developmental
coordination disorder: the mirror neuron system hypothesis. Journal of Behavioral and
Brain Science, 2(2), 258-268.
BIOLOGICAL AND ROBOTIC MODELS 12
Footnotes
1 Semantics is defined as the study of the meaning of a variety of languages (Thomason,
2012), and it can explain the mirroring of language communication with respect to mouth and
hand movements.
2 FARS is named after scientists Fagg, Arbib, Rizzolatti, and Sakata (Demiris, Aziz-
zadeh, & Bonaiuto, 2014).
3 F5 canonical neurons are responsible for learning particular skills like grasping (Demiris
et al., 2014).
4 HAMMER is an acronym for Hierarchical Attentive Multiple Models for Execution and
Recognition (Demiris et al., 2014).
5 Joint angle values are encoded motion elements in a robot (Dawood & Loo, 2016).
6 A natural scientist cannot fully understand the BAL Algorithm (Rebrová, Pecháč, &
Farkaš, 2013) without any conceptual knowledge of multivariable calculus, linear algebra, and
other advanced branches of mathematics.
BIOLOGICAL AND ROBOTIC MODELS 13
Figure 1. The temporo-parieto-premotor mirror pathway is fundamental to the normal workings
of the monkey MNS model (Rozzi, 2015).
BIOLOGICAL AND ROBOTIC MODELS 14
Figure 2. IPL, PMv, IFG, and STS are the primary regions within the mirror neuron system
(Werner, Cermak, & Aziz-Zadeh, 2012).
BIOLOGICAL AND ROBOTIC MODELS 15
Figure 3. FARS contains the reach (green box) and grasp (red box) circuit, which both lead to
grasping (Demiris et al., 2014).
BIOLOGICAL AND ROBOTIC MODELS 16
Figure 4. MNS performs a temporal-spatial transformation on hand state trajectory. It also
extends FARS for the reach and grasp circuits to interact with the core mirror neuron (blue box)
(Demiris et al., 2014).
BIOLOGICAL AND ROBOTIC MODELS 17
Figure 5. The MNS2 is the extension of MNS with the addition of hand and object working
memories (yellow box) and the primary and secondary auditory cortices (purple box) (Demiris et
al., 2014).
BIOLOGICAL AND ROBOTIC MODELS 18
Figure 6. HAMMER is arranged to support multiple perspectives toward action understanding,
including the generative (white and blue boxes) and passive (yellow box) routes. The flow of
information via the generative route can usually predict the nature of a particular action, whereas
the passive route records and replicates the demonstration while storing new inverse models for
the generative route to utilize in the future (Demiris et al., 2014).
BIOLOGICAL AND ROBOTIC MODELS 19
Figure 7. The humanoid MNS model is designed with such algorithms as the Topological
Gaussian Adaptive Resonance Hidden Markov Model (TGAR-HMM) for robotic imitation to
reflect human imitation while mirror neurons are activated (Dawood & Loo, 2016).
BIOLOGICAL AND ROBOTIC MODELS 20
Figure 8. Researchers investigate self-exploration as they use this architecture for imitative
learning, which starts with motion and vision processed by algorithms and joint angles (Dawood
& Loo, 2016).
BIOLOGICAL AND ROBOTIC MODELS 21
BIOLOGICAL AND ROBOTIC MODELS 22
Figure 9. With the addition of computational features like the BAL Algorithm and
SOMs/MSOMs, the robotic MNS model represents the biological mapping of mirror neurons
(Rebrová et al., 2013).
BIOLOGICAL AND ROBOTIC MODELS 23
Figure 10. The data collected from the EMG study shows that humanoid activity in imitating
humans and robots is comparable. However, when observing human and robot movement,
humanoid activation is greater in the former (Hofree et al., 2015).