Content uploaded by Carsten Giessing
Author content
All content in this area was uploaded by Carsten Giessing on Nov 25, 2014
Content may be subject to copyright.
Functional Interactions during the Retrieval
of Conceptual Action Knowledge: An fMRI Study
Ann Assmus
1,2
, Carsten Giessing
1
, Peter H. Weiss
1,2
,
and Gereon R. Fink
1,3
Abstract
& Impaired retrieval of conceptual knowledge for actions has
been associated with lesions of left premotor, left parietal, and
left middle temporal areas [Tranel, D., Kemmerer, D., Adolphs,
R., Damasio, H., & Damasio, A. R. Neural correlates of con-
ceptual knowledge for actions. Cognitive Neuropsychology,
409–432, 2003]. Here we aimed at characterizing the differential
contribution of these areas to the retrieval of conceptual
knowledge about actions. During functional magnetic reso-
nance imaging (fMRI), different categories of pictograms
(whole-body actions, manipulable and nonmanipulable objects)
were presented to healthy subjects. fMRI data were analyzed
using SPM2. A conjunction analysis of the neural activations
elicited by all pictograms revealed ( p < .05, corrected) a bi-
lateral inferior occipito-temporal neural network with strong
activations in the right and left fusiform gyri. Action picto-
grams contrasted to object pictograms showed differential
activation of area MT+, the inferior and superior parietal cortex,
and the premotor cortex bilaterally. An analysis of psycho-
physiological inte ractions identified contribution-dependent
changes in the neural responses when pictograms triggered
the retrieval of conceptual action knowledge: Processing of
action pictograms specifically enhanced the neural interaction
between the right and left fusiform gyri, the right and left middle
temporal cortices (MT+), and the left superior and inferior
parietal cortex. These results complement and extend previous
neuropsychological and neuroimaging studies by showing that
knowledge about action concepts results from an increased
coupling between areas concerned with semantic processing
(fusiform gyrus), movement perception (MT+), and temporo-
spatial movement control (left parietal cortex). &
INTRODUCTION
In 1870, Finkelnburg defined asymbolia as ‘‘a partial or
complete loss of the ability to comprehend and express
concepts by means of acquired signs.’’ He presented five
cases of aphasic patients who also showed nonverbal
symb olic impairments (e.g., for signs, notes, money,
rituals, or conventions). One patient (case II) exhibited,
in addition to her aphasia, impaired pantomime per-
formance on verbal command (e.g., making the sign of a
cross) but preserved imitation abilities (Duffy & Liles,
1979), whereas another global aphasic patient (case V)
showed a pantomime agnosia (Rothi, Mack, & Heilman,
1986). Finkelnburg concluded that asymbolia was the
underlying cause of these symptoms. Liepmann (1908),
in contrast, considered apraxia to be a disorder of skilled
movements and argued that apraxia might be more
apparent with symbolic gestures because they have to
be performed solely from memory without any help
from sensory information present during the man ip-
ulation of objects (see also Goldenberg, Hent ze, &
Hermsdo¨rfer, 2004).
Thus far, lesion studies failed to provide a clear answer
to this debate (Varney, 1982). Early studies (Goodglass &
Kaplan, 1963) showed that the gestural ability was not
related to the severity of aphasia when auditory com-
prehension was controlled for. In contrast, Kertesz and
Hooper (1982) found that apraxia correlated with severity
of aphasia and, in particular, with comprehension deficits.
However, the same group reported dissociations between
aphasia and apraxia (Kertesz, Ferro, & Shewan, 1984): Six
severely aphasic patients did not suffer from apraxia. In
contrast, all patients with severe apraxia (n = 40) also
showed severe aphasia (Kertesz et al., 1984). Recently,
using multidimensional scaling in 40 patients with left
brain damage, Goldenberg, Hartmann, and Schlott (2003)
failed to reveal a close clustering of deficits in pantomime,
imitation, drawing, and language tests (Goldenberg et al.,
2003). Similarly, Saygin, Wilson, Dronkers, and Bates
(2004) showed that in left-hemisphere-injured patients,
‘‘action comprehension deficits in the linguistic and non-
linguistic domains were not tightly correlated.’’
Nevertheless, studies concerned with the retrieval of
action knowledge independent of actual motor perform-
ance can help to disentangle the relationship between
impaired conceptual knowledge for actions and other
forms of symbolic impairments. For example, damage to
1
Research Center Ju¨lich, Germany,
2
University Hospital, RWTH
Aachen, Germany,
3
Cologne University Hospital, Germany
D 2007 Massachusetts Institute of Technology Journal of Cognitive Neuroscience 19:6, pp. 1004–1012
inferior occipito-temporal areas can lead to a dissocia-
tion between impaired recognition of line drawings of
objects and preserved recognition of action drawings
and pantomimes (Ferreira, Ceccaldi, Giusiano, & Poncet,
1998; Schwartz, Barrett, Crucia n, & Heilman, 1998).
These findings suggest that at least partially separable
neural networks underlying the processing of action and
object stimuli may exist, a notion which, on a more
general level, is consistent with the dual-route hypoth-
esis (Goodale & Milner, 1992) or the idea that concep-
tual knowledge is mediated by neural networks that
depend on sensorimotor attributes of the presented
information (Kable, Kan, Wilson, Thompson-Schil l, &
Chatterjee, 2005).
One group lesion study further explored knowledge
of action concepts by evaluating attributes of pictured
actions. In that study, the authors distinguished between
the retrieval of conce ptual knowledge (recognition) and
lexical retrieval (naming) and proposed ‘‘that action
concepts embody knowledge about the behaviours of
entities, especially animate entities such as people and
animals, but also inanimate entities such as tools and
vehicles’’ (Tranel, Kemmerer, Adolphs, Damasio, &
Damasio, 2003). These action concepts not only con-
tribute to the planning of movements but also to the re-
cognition of movements made by others (Buccino et al.,
2001). The highest lesion overlap in patients with im-
paired retrieval of conceptual knowled ge for actions
was found in the left premotor cortex, the left parietal
cortex, and in the white matter under neath the left
MT+. In contrast, impaired lexic al retrieval of action
knowledge was associated with left frontal opercular
lesions (Tranel, Adolphs, Damasio, & Damasio, 2001).
In this study, we aim to disentangle the specific con-
tribution of the areas subserving the re trieval of con-
ceptual knowledg e about actions as implicated by
the abovementioned lesion study (Tranel, Kemmerer,
Adolphs, Damasio, & Damasio, 2003). We refer to ac-
tion concepts as knowledge about complex actions that
contributes to the recognition of actions. To identify
the neural correlates of action knowledge, we compared
neural activity elicited by retrieving the meaning of ac-
tion pictograms to the neural activity eli cited by re-
trieving the meaning of pictograms not associated with
actions. Pictograms are visual symbols that are void of
any spoken or written information. Their meaning is
either related to the employed image or to a related
concept (Kitagami, Inoue, & Nishizaki, 2002). The action
pictograms employed in this study showed biological
movements which represented different kinds of sports.
To access the meaning of an action picto gram, knowl-
edge about the illustrated move ment had to be re-
trieved. Additionally, pictograms of different kinds of
objects were employed. We separ ated pictograms of
small, manipulable objects from large, nonmanipulable
objects, as nonmanipulable objects were not expected
to lead to any implicit action-related processing. With
respect to manipulable objects, it is still a matter of
debate whether action knowledge is a prerequisite for
the recognition of manipulable objects (Gerlach, Law,
Gade, & Paulson, 2002; Chao & Martin, 2000). Thus, we
presented different categories of black and whit e picto-
grams to healthy volunteers during functional magnetic
resonance imaging (fMRI): whol e-body a ctions, ma-
nipulable and nonmanipulable objects (see Figure 1).
During scanning, subjects were asked to indicate via
button press whether they knew the meaning of the
pictogram. For the processing of pictograms, irrespec-
tive of content, we expected activat ion of a network of
bilateral occipito-temp oral and left frontal brain areas,
comparable to activations observed in studies of se-
mantic retrieval with verbal stimuli (Noppeney, Phillips,
& Price, 2004). Due to its importance in object recogni-
tion, especially higher-level shape processing (Kourtzi
& Kanwisher, 2001), the lateral occipital complex of the
ventral stream was expected to be activated by picto-
g rams of manip ula ble and nonmanipulable objects
(Grill-Spector, Kourtzi, & Kanwisher, 2001). Furthermore,
additional activations of dorsal stream areas (premotor
and parietal cortex) could occur during process ing
of manipulable object pi ctograms (Beauchamp, Lee,
Haxby, & Martin, 2002; Chao & Martin, 2000). Processing
of act ion pictograms was expected to activate area
MT+, as Kable et al. (2005) and Kable, Lease-Spellmeyer,
and Chatterjee (2002) foun d differential activation of
human MT+ triggered by action pictures versus objec t
pictures in a picture–word matching task. Furthermore,
we expected action pictograms to lead to left parietal
cortex activation, an area recently shown to be involv ed
in retrieving knowledge about manip ulative actions
(Boronat et al., 2005).
Finally, we we re interested in assessing possible
changes of connectivity within the neural netw ork sub-
serving the processing of action and object pictograms.
Figure 1. Examples of pictogram stimuli: (A) action pictograms,
(B) manipulable objects, and (C) nonmanipulable objects.
Assmus et al. 1005
We according ly employed a design which allowed us, by
the analysis of psycho-physical interactions , to investi-
gate changes in the coupling between areas involved in
the processing of pictograms in general, and, more
importantly, with regard to the purpose of this study,
the areas specifically engaged in the retrieval of concep-
tual knowledge about actions.
METHODS
Subjects
Twelve health y, right-handed volunteers (5 w omen,
7 men; mean age (±SD) 25 ± 5 years) participated in
this fMRI experiment. Handedness was assessed using
the German version of the Edinburgh Handedness In-
ventory (mean laterality quotient = +84, SD = 16)
(Oldfield, 1971). No subject had any history of neuro-
logical or psychiatric illness. Informed written consent
was obtained from all subjects who were naı¨ve to the
purpose of the experiment. The study was approved by
the local ethics committee and was conducted in ac-
cordance with the Declaration of Helsinki.
Experimental Design
In this blocked fMRI experiment, we used three dif-
ferent categories of pictograms as stimuli (C1–C3) and
simple geometric objects for control (C4). B ecause
comparisons with a ‘‘resting state’’ can lead to artificial
task-dep endent de activations (McKiernan, Kaufman,
Kucera-Thompson, & Binder, 2 003), we introduced
the control condition (C4) with comparable visual in-
put, response selection, and identical motor responses
to Conditions 1–3, but without processing of complex
pictograms (i.e., only squares or circles were shown).
Stimuli were presented using Presentation 0.76 (Neuro-
behavioral Systems, California) software and projected
on a screen from a distance of 29 cm to the subject
through a mirror fixed to the head coil. Subjects were
allowed to move their eyes to avoid an interaction of the
neural mechanisms underlying covert attention with the
neural p rocesses of interest (Fink, Dolan, Halligan,
Marshall, & Frith, 1997).
The sti muli used were pictograms symbolizing (a)
whole-body actions (C1), (b) manipulable objects (C2),
(c) nonmanipulable objects (C3), or (d) squares /circles
(C4, for control) (see Figure 1). Pictograms are graphi-
cal symbols frequent ly used at public facilitie s, such
as public transportation, sports venues, or commercial
facilities, and represent either a concept or an object by
illustration (see, e.g., www.ecomo.or.jp/symbols_english/
symbol_index2.html). A standard set of pictograms is
defined in the international standard ISO 7001: Public
Information Symbols. They can be an effective means
of providing important information without language.
All pictograms used in this study were black on a white
background. For the action condition (C1), sport-related
pictograms were used (Figure 1A). For the manipulable
objects condition (C2), pictograms of man-made artifacts
were shown, some of which were tools (Figure 1B). For
the nonmanipulable objects condition (C3), large entities
such as buildings, vehicles, but also pictograms depict-
ing animals (e.g., indicating a farm) or static humans (e.g.,
indicating a restroom), were used (Figure 1C). For con-
trol (C4), squares and circles were presented to the sub-
jects (8 squares, 2 circles per block in random order).
The central projection of the stimuli on the white
screen subtended a vertical and horizontal visual angle
up to 5.48. Stimuli were shown fo r 2400 msec. The
interstimulus interval was jittered and was either 300,
600, or 900 msec, during which a blank screen was
sh own. The stimuli were grouped according to the
condition of interest in blocks of 30 sec, each consisting
of 10 stimuli. The blocks were separated by a 20-sec
baseline (blank screen), which was followed by an
instruction of 5 sec announcing the upcoming condi-
tion. In C1 to C3, subjects indicated by but ton presses
whether they knew the meaning of the symbol. In C4,
subjects indicated by button presses whether a square
was presented . Subjects responded by pressing t wo
keys with either the index or middle finger of their
right/left hand indicating ‘‘yes’’ with their index and
‘‘no’’ with their middle finger. In half of the trials,
subjects used their right hand, in the other half, subjects
used their left hand.
fMRI images were acquired during two sessions of
15 min each. This experiment consisted of 32 blocks,
yielding a total of eight blocks (80 trials) per condition.
Conditions were presented in counterbalanced order
within and between subjects, and trials within a block
appeared in a randomized order. Prior to the experimen-
tal session, subjects were informed that they had to fill
in a questionnaire afterward, in which they should name
the presented symbols. These questionnaires consisted
of all 240 presented pictogram stimuli. Subjects were
asked to describe each symbol’s meaning in one word.
Behavioral Data Analysis
Re act ion times were acquired during scan ning. The
questionnaires filled in by the subjects after scanning
were tested for omissions and errors. Repeated-measures
analysis of variance (ANOVA) in SPSS was used for the
analysis of the behavioral data.
Structural and Functional Magnetic
Resonance Imaging
fMRI images were acquired on a Siemens Sonata 1.5-T
wh ole -bo dy scanner with e cho -pl ana r imaging (EPI)
capability using th e standard radio-freq uency head
coil. Multislice T2*- weighted echo-planar images were
obtained from a gradient-echo sequence with the fol-
1006 Journal of Cognitive Neuroscience Volume 19, Number 6
lowing parameters: echo time (TE) = 66 msec, repe-
tition time (TR) = 2.5 sec, flip angle = 908, field of
view (FOV) = 200 mm, slice thickness = 4 mm, in-
terslice gap = 0.4 mm, in-plane resolution = 3.125 !
3.125 mm
2
. The 24 slices were aligned to the AC–PC
line. To obtain anatomical images at high resolution, a
T1-weighted MP RAGE (magnetization-prepared, rapid
acquisition gradient echo) sequence was used (TE =
3.93 msec, TR = 2.2 msec, in version time TI = 1200, flip
angle 158 , FOV = 240 mm, slice thickness = 1.5 mm,
matrix = 180 ! 256, number of sagittal slices = 128).
Image Processing
After applying standard procedures for image realign-
ment, slice timing, normalization (voxel size 3 ! 3 !
3 mm), and smoothing, data were analyzed with Statistical
Parametric Mapping software SPM2 (Wellcome Depart-
ment of Imaging Neuroscience, London, UK; www.fil.ion.
ucl.ac.uk) using a random effects analysis. After spatial
transformation, the functional data were smoothed with a
Gaussian kernel of 8 mm to reduce the variance due to
anatomical variability. The time series were high-pass
filtered (with a cutoff frequency of 1/128 Hz) to remove
low-frequency artifacts. We included six head movement
parameters as regressors to control for movement-relat-
ed variance. The different conditions were modeled as
box-car functions convolved with the canonical synthetic
hemodynamic response function in SPM2. Specific ef-
fects were tested by applying appropriate linear con-
trasts to the parameter estimates for each condition,
resulting in a t statistic for each voxel. An activation
cluster was considered significant when it passed a
threshold of p < .05, corrected at the cluster level (with
p < .001 uncorrected at the voxel level).
For the conjunction analysis, a one-way ANOVA in
SPM2 and the revised test proposed by Nichols, Brett,
Andersson, Wager, and Poline (2004) with a voxel level
threshold of p < .05, familywise error (FWE) corrected
(no cluster threshold possible in a conjunction analysis),
were applied. Only clusters extending to a size of at least
20 voxels are reported. The conjunction analysis of all
three pictogram conditi ons > control was calculated to
reveal common activations across the different catego-
ries of pictograms. The contrasts manipulable objects >
nonmanipulable objects and nonmanipulable objects >
manipulable objects were used to test for differences
between the processing of the two object categories.
The contrast action pictograms > objects (manipulable
and nonmanipulable combined) was calculated to look
for action-specific activations.
Finally, psycho-physiological interactions (PPIs) (Friston
et al., 1997) were calculated for the two largest clusters
(rig ht and left fusiform gyrus) elicited by pictogram
processing (irrespective of pictogram type as revealed
by the conjunction analysis). The coordinates of the
voxels of maximal activation intensity in the contrast pic-
tograms > control for the group (i.e., in the right and left
fusiform gyrus) were used in the contrast pictograms >
control at the single-subject level. A sphere of 4 mm
radius was centered at the nearest local maximum. The
mean-corrected blood oxygen level-dependent signal
time course from this sph ere was then used as the
physiological factor. The psychological factor was a vector
coding for the main effect of symbol category (2 for action
symbols, "1 for manipulable objects, "1 for nonmanipu-
lable objects) convolved with the hemodynamic response
function. The PPI was then computed as the element-by-
element product of the physiological and the psycholog-
ical vectors. For each subject, we created a new statistical
model containing the PPI as regressor and the psycho-
logical and the physiological vectors as covariates of no
interest. Subjects’ specific contrast images were then en-
tered into random effects group analyses. The statistical
threshold was again set at p < .05, corrected at the cluster
level (with p < .001 uncorrected at the voxel level).
RESULTS
Behavioral Data
The action pictograms evoked significantly longer re-
action times than the (manipulable and nonmanipula-
ble) object picto grams and the control condition [mean
RT in msec ± SD: actions 1176 msec ± 281 msec, ob-
jects 1082 msec ± 270 msec, control 610 ± 106 msec;
F(2, 10) = 28,55, p < .05].
The analysis of the questionnaires showed a signifi-
cant effect of condition [mean error rate in % ± SD:
actions: 5 ± 3%, objects: 2 ± 2%, F(2, 10) = 9.72, p < .05]
as significantly more action pictograms were misinter-
preted compared to the (manipulable and nonmanipu-
lable) objec t conditions. The rate of omissions was not
significantly different between conditions (mean rate of
omissions in % ± SD: actions 2 ± 2%, objects 2 ± 3%).
Neural Activations
Retrieval of Pictogram Meaning
The retrieval of pictogram meaning (i.e., pictograms rep-
resenting both actions and objects, manipulable as well
as nonmanipulable) contrasted to the control condition
(squares , circles) revealed (as depicted by the con-
junction analysis) bilateral activation of the inferior
occipito-temporal cortex including the fusiform gyrus
(see Figure 2A).
The contrasts manipulable objects > nonmanipulable
obje cts and nonmanipulable objects > manipulable
objects led to no significant differential activations. Thus,
the two object conditions were collapsed in all subse-
quent analyses.
The contrast action pictograms > object pictograms
revealed differential activations in the superior temporal
sulcus, extending into the human motion-sensitive area
Assmus et al. 1007
MT+ (Malikovic et al., 2007; Zeki et al., 1991), in the su-
perior and inferior parietal cortex, and the premotor cor-
tex bilaterally (see Figure 2B). With the exception of area
MT+, these areas did not show differential activations
between the object and control conditions (see Figure
2C). The reverse contrast (object pictograms > action
pictograms) showed no significant differential activations.
Psycho-physiological Interactions
The activation peaks of the two largest clusters in the
right (+45, "60, "15) and left ("39, "72, "15) fusiform
gyrus (as revealed by the conjunction analysis of picto-
gram processing irrespective of category) were chos en
for the analysis of PPIs. We tested for possible functional
interactions of the left and right fusiform gyri with other
brain regions that may critically depend on the category
of the processed pictogram. For the right fusiform gyrus
(45, "60, "12), significant context-dependent contribu-
tions during the pr ocessing of action pictograms were
found in the right middle temporal (MT+; 57, "66, 6;
Z = 3.91), the left superior pari etal ("9, "63, 66;
Z = 4.6), and the left inferior pariet al cortex, including
the left supramarginal gyrus, ("54, "33, 30; Z = 3.67; all
Figure 2. Neural activations
related to the processing
of pictograms in general
(A) and of action pictograms
specifically (B) as well as the
mean signal changes for the
four experimental conditions
in the main areas of the left
hemisphere (C). (A) Retrieving
the meaning of object and
action pictograms (each
pictogram type contrasted
to the control condition in a
conjunction analysis) activated
the inferior occipito-temporal
cortex, including the fusiform
gyrus bilaterally. (B) The
neural activations associated
with processing of action
pictograms (contrasted to
object pictograms) included
the human MT+ bilaterally,
the superior and inferior
parietal cortices bilaterally, and
the premotor cortex bilaterally.
(C) Mean signal change in the
left MT+ ("54, "66, +3), left
superior parietal lobe (SPL;
"24, "63, +63), left inferior
parietal lobe (IPL; "54, "36,
+27), and left premotor cortex
(PMC; "30, "3, +63) for the
action pictograms (AP),
pictograms of manipulable
(MOP) and nonmanipulable
objects (NMOP), and the
control condition
(C, geometric shapes),
respectively.
1008 Journal of Cognitive Neuroscience Volume 19, Number 6
p < .05, corrected) (see Figure 3). No significant PPIs
were found during the processing of pictograms repre-
senting objec ts (for ma nipulable and nonmanipulable
objects combined, as no significant differenti al activa-
tions for the two object categories had been observed,
see above).
For the left fusiform gyrus ("42, "69, "15), signifi-
cant context-dependent contributions during the pro-
cessing of action pictograms were found in the right and
left middle temporal (MT+; 51, "66, 6; Z = 4.4; "57,
"63, 9; Z = 4.22) and the left superior parietal cortex
("12, "69, 63; Z = 4.43; all p < .05, corrected) (see
Figure 3). No significant changes in the contribution of
the left fusiform gyrus were observed during the o bject
conditions.
DISCUSSION
The aim of this study was to explore further the contri-
bution of the areas involved in the neural network that
underlies the retrieval of conceptual action knowledge
as triggered by action pictograms. Pictograms are sym-
bols which represent a concept by illustration. In order
to recognize the meaning of a pictogram, knowledge
ab out the underlying concept is a prerequisite. We
explored the brain areas differentially recruited during
the processing of action, but not object pictograms. To
achieve this, we first detected by a conjunction analysis
the neural network that was active during all pictogram
conditions, and then looked for differential activations
during the processing of action pictograms (relative to the
processing of object pictograms). Finally, using psycho-
physical interactions, we explored enhanced neural in-
teractions between areas generally involved in semantic
processing of pictograms and the network supporting
action concept retrieval (specifically) triggered by action
pictograms.
Processing of pictograms irrespective of pictogram type
(actions as well as manipulable and nonmanipulable ob-
jects) led to activations in the inferior occipito-temporal
cortex bilaterally, as determined by a conjunction analysis.
The peak activations within the large occipito-temporal
clusters were located in the right and left fusiform gyri.
These areas have previously been associated with se-
mantic retrieval of word stimuli (Noppeney et al., 2004;
Wagner, Pare-Blagoev, Clark, & Poldrack, 2001) and with
semantic memory in general (Martin & Chao, 2001).
The retrieval of action concepts triggered by action
pictograms compared to the processing of the meaning
of object pictograms activated a neural network consist-
ing of MT+, the superior and inferior pariet al cortex,
and the premotor cortex bilaterally. Thus, areas con-
cerned with movement perception (Assmus et al., 2003;
Zeki et al., 1991) and execution (Hermsdorfer et al.,
2001) were also active during the processing of static
action pictograms (Kourtzi & Kanwisher, 2000). That the
retrieval of action and object concepts, which contribute
to the recognition of the respective pictograms, may rely
on differential neural substrates is supported by neuro-
psychological single-case studies of dissoci ations be-
tween action and object recognition deficits (Magnie,
Ferreira, Gius iano, & Poncet, 1999; Ferreira et al., 1998;
Schwartz et al., 1998). Furthermore, investigat ing the
neural responses to overt motion of humans and (ma-
nipulable) objects, Beauchamp et al. (2002) found seg-
regated responses to human and object stimuli in the
posterior temporal cortex.
A possible confound in the comparison between
action and object pictogram s is the fact tha t only the
former stimuli included human-like forms engaged in
whole-body actions. The perception of humans (Saxe,
Figure 3. Psycho-
physiological interactions.
During processing of action
pictograms, the right fusiform
gyrus showed significant
interactions with the left
superior and inferior parietal
cortex (supramarginal gyrus)
and right MT+, whereas the
left fusiform gyrus significantly
interacted with the left
superior parietal cortex
and MT+ bilaterally.
Assmus et al. 1009
Jamal, & Powell, 2006) and inferring intentions from hu-
man actions (Saxe, Xiao, Kovacs, Perrett, & Kanwisher,
2004) activates, at least in part, areas that we also found
for the categorical comparison between the processing
of action and object pictograms. It should be noted,
however, that Saxe and coworke rs used pictures of
real humans, whereas we used pictograms of sportive
whole-body actions. Because our study design did not
contain person-without-action pictograms, we cannot dif-
ferentiate whether activity in, for instance, the posterior
temporal cortex, foun d in the current study, represents
conceptual knowledge about human actions, perception
of humans per se, or a combination thereof. This is an
interesting question for further studies poss ibly using
parametric designs in which the amount of implied
(human) action is systematically varied.
For the categorical comparison between the process-
ing of action and object pictograms, the two object
pictogram types were collapsed, as the differential con-
trasts between the manipulable and the nonmanipulable
object conditions reveal ed no significant di fferences.
The lack of differential neural activations for manipula-
ble and nonmanipulable object pictograms was some-
what unexpected considering previou s studies, which
point to an important role of action knowledge in the
processing of manipulable objects (and tools) (Grezes &
Decety, 2002; Chao & Martin, 2000; Tranel, Damasio, &
Damasio, 1997). A possible explanation for this negative
result could be related to the task used in our study:
Subjects were requ ired to retrieve the meaning of the
object pictograms, but not to categorize them. In fact,
Gerlach et al. (2002) found activation of the left premo-
tor cortex only during categorization of tools, but not
during the mere naming of tools. Further evidence that
processing of object stimuli is influenced by the respec-
tive task requirements is provided by a study of Valyear,
Culham, Sharif, Wes twood, and Goodale (2006 ), in
which the perception of an identical object differen-
tially activated dorsal or ventral stream areas depending
on whether subjects processed the object’s orientation
(implicitly activating the appropriate grasp for the ob-
ject) or its identity. Therefore, the lack of dorsal stream
activations when processing pictograms of manipulable
objects (compar ed to nonmanipulable object picto-
grams) could be due to the fact that our subjects were
instructed not to consider the actions related to the re-
spective objects, but only to retrieve the meaning of the
object pictograms. The differential neural systems for
nouns and verbs (Damasio & Tranel, 1993) may provide
another explanation for the lack of difference between
the manipulable and nonmanipulable object conditions.
Because all objects are denoted by nouns, the retrieval
of the pictogram meaning may have (implicitly) activat-
ed the neural representations of verbs (for action picto-
grams) and nouns (for both the manipulable and the
nonmanipulable object conditions). This interpretation
would be consistent with the notion ‘‘that noun–verb
dissociations reflect salient differences in the neural rep-
resentation of objects and actions’’ and ‘‘that the brain
distinguishes between nouns and verbs on the basis of
semantics (meaning)’’ (Shapiro et al., 2005, p. 1058).
The right and left fusiform gyri, the areas maximally
activated in the conjunction analysis of processing of
all pictogram types, were chosen as the seeds for the
analysis of PPI s. This analysis tested whether any region
throughout the whole brain showed context-depend ent
changes in coupling with the right and left fusiform gyri.
Interestingly, only during the retrieval of conceptua l
action knowledge triggered by action pictograms were
significant PPIs detected. These PPIs consisted of an in-
creased coupling between the fusiform gyri and the mid-
dle temporal cortex (MT+) bilaterally as well as the left
superior and left inferior parietal cortex. Thus, the areas
revealed by the PPI analysis were parts of the larger
network determined by the categorical comp arison of
action versus object pictogram processing, which addi-
tionally included the right superior and inferior parietal
cortex and the bilateral premotor cortex. Taken to-
gether, our data suggest that within a larger network
concerned with the processing of action stimuli, the
retrieval of conceptual action knowledge is realized by
an increased coupling between areas concerned with se-
mantic processing (fusiform gyrus), perception of move-
ment (MT+), and temporo-s patial movement control
(left parietal cortex).
Our findings are in good accordance with neuropsy-
chological studies: Patients with impaired knowledge for
actions showed a lesion overlap in the white matter
underneath the left MT+, in the left parietal cortex, and
in the left premotor/prefrontal cortex ( Tranel e t al.,
2003). The authors proposed that, ‘‘when we evoke
the conce pt of an action, we activate collec tions of
sensory and motor patterns in cerebral cortices appro-
priate to represent pertinent features of the concept.’’
The human MT+ is known to be engaged not only in
overt motion processing (Zeki et al., 1991) but also in
processing impl ied motion from static images (Kourtzi &
Kanwisher, 2000; Peigneux et al., 2000). Therefore, the
bilateral activation of MT+ in the current study is likely
to represent the implied motion related to the retrieved
action concepts. Activation of MT+ duri ng access of
action knowledge has already been shown in previous
neuroimaging studies (Kable et al., 2002). Kable et al.
(2002) compared conceptual matching of actions to
conceptual matching of objects, and found activations
in occipito-temporal regions ne ar MT+. Kable et al.
concluded that regions around MT+ process both con-
ceptual and perceptual features of motion.
The importance of the left parietal cortex in the con-
trol of complex action is undisputed (Liepmann, 1905).
Although the left inferior parietal cortex is involved in
temporo-spatial movement control (Assmus, Marshall,
Noth, Zilles, & Fink, 2005; Assmus et al., 2003; Weiss
et al., 2001; Poizner et al., 1998; Liepmann, 1905), the
1010 Journal of Cognitive Neuroscience Volume 19, Number 6
left superior parietal cortex contributes to the internal
representations used for the control of actions (Wolpert,
Goodbody, & Husain, 1998). Thus, activation of the left
parietal cortex during the retrieval of action concepts
is likely to represent aspects of higher motor control.
Furthermore, this finding corresponds well to neuropsy-
chological studies, which demonstra te the importance
of the left parietal cortex for gesture recognition (Varney
& Damasio, 1987; Ferro, Martins, Mariano, & Caldas,
1983). Based on the results of their Gesture Recognition
Test in 65 patients with left hemisphere lesions, Ferro
et al. (1983) suggested that gestures are represented at
the conceptual–symbolic level in the (left) parietal cor-
tex. With respect to our results of the analysis of PPIs, it
is noteworthy that Varney and Damasio (1987) sug-
gested that the supramarginal, rather than the angular,
gyrus is involv ed in pantomime recognition.
Taken together, the analysis of the PPIs revealed that
both semantic areas (fusiform gyrus) and areas concerned
with the perception (MT+) and control of actions (left
parietal cortex) are involved in accessing conceptual
action knowledge. Our results are in line with conclusions
drawn by Buxbaum, Schwartz, and Carew (1997) con-
cerning the role of the semantic system for object use.
The authors hypothesized that the execution of complex
goal-directed movements is facilitated by a linkage be-
tween semantic and se nsorimotor information about
objects. On the other hand , preserved sensorimotor
experiences triggered by object use may support recog-
nition of (manipulable) objects even in severe semantic
agnosia (Magnie et al., 1999). By showing that accessing
conceptual information about motion attributes activates
the middle temporal cortex (MT+), Kable et al. (2005)
also supported the view that ‘‘conceptual knowledge is
instantiated by distributed neural regions partially orga-
nized along sensorimotor lines.’’
In summary, we found that knowledge about action
concepts is retrieved by the interaction between areas
engaged during semantic processing and a network of
sensorimotor areas. With respect to the clinical obser-
vations made by Finkelnburg (1870) concerning asym-
bolia, our data suggest that patients suffering from a
disconnection syndrome between semantic and senso-
rimotor brain re gions are likely to show deficits in the
understanding and expression of symbolic actions.
Acknowledgments
We thank all our volunteers. We also thank our colleagues
from the MR and Cognitive Neurology group for both technical
support and many fruitful discussions. G. R. F. is supported by
the DFG (KFO-112, TP1), Germany.
Reprint requests should be sent to Ann Assmus, Institute of
Neuroscience and Biophysi cs—Medicine, Research Centre
Ju¨lich, 52425 Ju¨lich, Germany, or via e-mail: a.assmus@fz-
juelich.de.
REFERENCES
Assmus, A., Marshall, J. C., Noth, J., Zilles, K., & Fink, G. R.
(2005). Difficulty of perceptual spatiotemporal integration
modulates the neural activity of left inferior parietal cortex.
Neuroscience, 132, 923–927.
Assmus, A., Marshall, J. C., Ritzl, A., Zilles, K., Noth, J., & Fink,
G. R. (2003). Left inferior parietal cortex integrates time
and space during collision judgements. Neuroimage, 20,
S82–S88.
Beauchamp, M. S., Lee, K. E., Haxby, J. V., & Martin, A. (2002).
Parallel visual motion processing streams for manipulable
objects and human movements. Neuron, 34, 149–159.
Boronat, C. B., Buxbaum, L. J., Coslett, H. B., Tang, K., Saffran,
E. M., Kimberg, D. Y., et al. (2005). Distinctions between
manipulation and function knowledge of objects: Evidence
from functional magnetic resonance imaging. Behavioral
Brain Research, 23, 361–373.
Buccino, G., Binkofski, F., Fink, G. R., Fadiga, L., Fogassi, L.,
Gallese, V., et al. (2001). Action observation activates
premotor and parietal areas in a somatotopic manner:
An fMRI study. European Journal of Neuroscience, 13,
400–404.
Buxbaum, L. J., Schwartz, M. F., & Carew, T. G. (1997). The
role of semantic memory in object use. Cognitive
Neuropsychology, 14, 219–254.
Chao, L. L., & Martin, A. (2000). Representation of manipulable
man-made objects in the dorsal stream. Neuroimage, 12,
478–484.
Damasio, A. R., & Tranel, D. (1993). Nouns and verbs are
retrieved with differently distributed neural systems.
Proceedings of the National Academy of Sciences, U.S.A., 90,
4957–4960.
Duffy, R. J., & Liles, B. Z. (1979). A translation of Finkelnburg’s
(1870) lecture on aphasia as ‘‘asymbolia’’ with commentary.
Journal of Speech and Hearing Disorders, 44, 156–168.
Ferreira, C. T., Ceccaldi, M., Giusiano, B., & Poncet, M. (1998).
Separate visual pathways for perception of actions and
objects: Evidence from a case of apperceptive agnosia.
Journal of Neurology, Neurosurgery, and Psychiatry, 65,
382–385.
Ferro, J. M., Martins, I. P., Mariano, G., & Caldas, A. C. (1983).
CT scan correlates of gesture recognition. Journal of
Neurology, Neurosurgery, and Psychiatry, 46, 943–952.
Fink, G. R., Dolan, R. J., Halligan, P. W., Marshall, J. C.,
& Frith, C. D. (1997). Space-based and object-based visual
attention: Shared and specific neural domains. Brain, 120,
2013–2028.
Finkelnburg, D. C. (1870). Votrag vor der Niederrheinischen
Gesellschaft (Sitzung vom 21. Ma¨rz 1870 in Bonn). Berliner
Klinische Wochenschrift, 7, 449–450.
Friston, K. J., Buechel, C., Fink, G. R., Morris, J., Rolls, E., &
Dolan, R. J. (1997). Psychophysiological and modulatory
interactions in neuroimaging. Neuroimage, 6, 218–229.
Gerlach, C., Law, I., Gade, A., & Paulson, O. B. (2002). The
role of action knowledge in the comprehension of
artefacts—A PET study. Neuroimage, 15, 143–152.
Goldenberg, G., Hartmann, K., & Schlott, I. (2003). Defective
pantomime of object use in left brain damage: Apraxia
or asymbolia? Neuropsychologia, 41, 1565–1573.
Goldenberg, G., Hentze, S., & Hermsdo¨rfer, J. (2004). The
effect of tactile feedback on pantomime of tool use in
apraxia. Neurology, 63, 1863–1867.
Goodale, M. A., & Milner, A. D. (1992). Separate visual
pathways for perception and action. Trends in
Neurosciences, 15, 20–25.
Goodglass, H., & Kaplan, E. (1963). Disturbance of gesture
and pantomime in aphasia. Brain, 86, 703–720.
Assmus et al. 1011
Grezes, J., & Decety, J. (2002). Does visual perception of
object afford action? Evidence from a neuroimaging study.
Neuropsychologia, 40, 212–222.
Grill-Spector, K., Kourtzi, Z., & Kanwisher, N. (2001). The
lateral occipital complex and its role in object recognition.
Vision Research, 41, 1409–1422.
Hermsdorfer, J., Goldenberg, G., Wachsmuth, C., Conrad, B.,
Ceballos-Baumann, A. O., Bartenstein, P., et al. (2001).
Cortical correlates of gesture processing: Clues to the
cerebral mechanisms underlying apraxia during the imitation
of meaningless gestures. Neuroimage, 14, 149–161.
Kable, J. W., Kan, I. P., Wilson, A., Thompson-Schill, S. L., &
Chatterjee, A. (2005). Conceptual representations of action
in the lateral temporal cortex. Journal of Cognitive
Neuroscience, 17, 1855–1870.
Kable, J. W., Lease-Spellmeyer, J., & Chatterjee, A. (2002).
Neural substrates of action event knowledge. Journal
of Cognitive Neuroscience, 14, 795–805.
Kertesz, A., Ferro, J. M., & Shewan, C. M. (1984). Apraxia
and aphasia: The functional–anatomical basis for their
dissociation. Neurology, 34, 40–47.
Kertesz, A., & Hooper, P. (1982). Praxis and language: The
extent and variety of apraxia in aphasia. Neuropsychologia,
20, 275–286.
Kitagami, S., Inoue, T., & Nishizaki, Y. (2002). Information
processing of pictograms and the visual field difference.
Perceptual and Motor Skills, 95, 173–183.
Kourtzi, Z., & Kanwisher, N. (2000). Activation in human
MT/MST by static images with implied motion. Journal
of Cognitive Neuroscience, 12, 48–55.
Kourtzi, Z., & Kanwisher, N. (2001). Representation of
perceived object shape by the human lateral occipital
complex. Science, 293, 1506–1509.
Liepmann, H. (1905). Die linke Hemispha¨re und das Handeln.
Mu¨nchener Medizinische Wochenschrift, 52, 232 und 2375.
Liepmann, H. (1908). Drei Aufsa¨tze aus dem Apraxiegebiet.
Berlin: Karger.
Magnie, M.-N., Ferreira, C. T., Giusiano, B., & Poncet, M.
(1999). Category specificity in object agnosia: Preservation
of sensorimotor experiences related to objects.
Neuropsychologia, 37, 67–74.
Malikovic, A., Amunts, K., Schleicher, A., Mohlberg, H.,
Eickhoff, S. B., Wilms, M., et al. (2007). Cytoarchitectonic
analysis of the human extrastriate cortex in the region
of V5/MT+: A probabilistic, stereotaxic map of area hOc5.
Cerebral Cortex, 17, 562–574.
Martin, A., & Chao, L. L. (2001). Semantic memory and the
brain: Structure and processes. Current Opinion in
Neurobiology, 11, 194–201.
McKiernan, K. A., Kaufman, J. N., Kucera-Thompson, J., &
Binder, J. R. (2003). A parametric manipulation of factors
affecting task-induced deactivation in functional
neuroimaging. Journal of Cognitive Neuroscience, 15,
394–408.
Nichols, T., Brett, M., Andersson, J., Wager, T., & Poline, J. B.
(2004). Valid conjunction inference with the minimum
statistic. Neuroimage, 25, 653–660.
Noppeney, U., Phillips, J., & Price, C. (2004). The neural
areas that control the retrieval and selection of semantics.
Neuropsychologia, 42, 1269–1280.
Oldfield, R. C. (1971). The assessment and analysis of
handedness: The Edinburgh Inventory. Neuropsychologia,
9, 97–113.
Peigneux, P., Salmon, E., Van der Linden, M., Garraux, G.,
Aerts, J., Delfiore, G., et al. (2000). The role of lateral
occipitotemporal junction and area MT/V5 in the visual
analysis of upper-limb postures. Neuroimage, 11, 644–655.
Poizner, H., Clark, M. A., Merians, A. S., Macauley, B., Rothi,
L. J. G., & Heilman, K. M. (1998). Left hemispheric
specialization for learned, skilled, and purposeful action.
Neuropsychology, 12, 163–182.
Rothi, L. J. G., Mack, L., & Heilman, K. M. (1986). Pantomime
agnosia. Journal of Neurology, Neurosurgery, and
Psychiatry, 49, 451–454.
Saxe, R., Jamal, N., & Powell, L. (2006). My body or yours? The
effect of visual perspective on cortical body representations.
Cerebral Cortex, 16, 178–182.
Saxe, R., Xiao, D.-K., Kovacs, G., Perrett, D. I., & Kanwisher,
N. (2004). A region of right posterior superior temporal
sulcus responds to observed intentional actions.
Neuropsychologia, 42, 1435–1446.
Saygin, A. P., Wilson, S. M., Dronkers, N. F., & Bates, E. (2004).
Action comprehension in aphasia: Linguistic and
non-linguistic deficits and their lesion correlates.
Neuropsychologia, 42, 1788–1804.
Schwartz, R. L., Barrett, A. M., Crucian, G. P., & Heilman, K. M.
(1998). Dissociation of gesture and object recognition.
Neurology, 50, 1186–1188.
Shapiro, K. A., Mottaghy, F. M., Schiller, N. O., Poeppel, T. D.,
Flu¨ß, M. O., Mu¨ller-Ga¨rtner, H.-W., et al. (2005). Dissociating
neural correlates for nouns and verbs. Neuroimage, 24,
1058–1067.
Tranel, D., Adolphs, R., Damasio, H., & Damasio, A. R. (2001).
A neural basis for the retrieval of words for actions.
Cognitive Neuropsychology, 18, 655–670.
Tranel, D., Damasio, H., & Damasio, A. R. (1997). A neural basis
for the retrieval of conceptual knowledge.
Neuropsychologia, 35, 1319–1327.
Tranel, D., Kemmerer, D., Adolphs, R., Damasio, H., &
Damasio, A. R. (2003). Neural correlates of conceptual
knowledge for actions. Cognitive Neuropsychology, 20,
409–432.
Valyear, K. F., Culham, J. C., Sharif, N., Westwood, D., &
Goodale, M. A. (2006). A double dissociation between
sensitivity to changes in object identity and object
orientation in the ventral and dorsal visual streams:
A human fMRI study. Neuropsychologia, 44, 218–228.
Varney, N. R. (1982). Pantomime recognition defect in aphasia:
Implications for the concept of asymbolia. Brain and
Language, 15, 32–39.
Varney, N. R., & Damasio, H. (1987). Locus of lesion in
impaired pantomime recognition. Cortex, 23, 699–703.
Wagner, A. D., Pare-Blagoev, E. J., Clark, J., & Poldrack, R. A.
(2001). Recovering meaning: Left prefrontal cortex guides
controlled semantic retrieval. Neuron, 31, 329–338.
Weiss, P. H., Dohle, C., Binkofski, F., Schnitzler, A., Freund, H.,
& Hefter, H. (2001). Motor impairment in patients with
parietal lesions: Disturbances of meaningless arm movement
sequences. Neuropsychologia, 39, 397–405.
Wolpert, D. M., Goodbody, S. J., & Husain, M. (1998).
Maintaining internal representations: The role of the human
superior parietal lobe. Nature Neuroscience, 1, 529–533.
Zeki, S., Watson, J. D. G., Lueck, C. J., Friston, K. J., Kennard,
C., & Frackowiak, R. S. J. (1991). A direct demonstration
of functional specialization in human visual cortex. Journal
of Neuroscience, 11, 641–649.
1012 Journal of Cognitive Neuroscience Volume 19, Number 6
