Sequences of Abstract Nonbiological Stimuli Share Ventral Premotor Cortex with Action Observation and Imagery

Abstract

Activation triggered by either observed or imagined actions suggests that the ventral premotor cortex (PMv) provides an action vocabulary that allows us to detect and anticipate basically invariant perceptual states in observed actions. In the present study, we tested the hypothesis that the same PMv region is also recruited by nonbiological (abstract) stimulus sequences as long as the temporal order of stimuli has to be processed. Using functional magnetic resonance imaging, we instructed participants to assess expected outcomes in observed actions [external biological cues (EB)], motor imagery [internal biological cues (IB)], or geometrical figure sequences [external nonbiological cues (EN)]. As hypothesized, we found that each condition elicited significant activation within PMv [left hemisphere, Brodman Area (BA) 6], in contrast to a sequential target detection control task. In addition, cue-specific activations were identified in areas that were only engaged for biologically (action) cued (EB, IB) and nonbiologically cued (EN) tasks. Biologically cued tasks elicited activations within inferior frontal gyri adjacent to PMv (BA 44/45), in the frontomedian wall, the extrastriate body area, posterior superior temporal sulci, somatosensory cortices, and the amygdala-hippocampal-area, whereas the nonbiologically cued task engaged presupplementary motor area, middle frontal gyri, intraparietal sulci, and caudate nuclei of the basal ganglia. In sum, findings point to a basic premotor contribution to the representation or processing of sequentially structured events, supplemented by different sets of areas in the context of either biological or nonbiological cues.

Full text

Behavioral/Systems/Cognitive
Sequences of Abstract Nonbiological Stimuli Share Ventral
Premotor Cortex with Action Observation and Imagery
Ricarda I. Schubotz and D. Yves von Cramon
Department of Neurology, Max-Planck-Institute for Human Cognitive and Brain Sciences, 04103 Leipzig, Germany
Activation triggered by either observed or imagined actions suggests that the ventral premotor cortex (PMv) provides an action vocab-
ulary that allows us to detect and anticipate basically invariant perceptual states in observed actions. In the present study, we tested the
hypothesis that the same PMv region is also recruited by nonbiological (abstract) stimulus sequences as long as the temporal order of
stimuli has to be processed. Using functional magnetic resonance imaging, we instructed participants to assess expected outcomes in
observed actions [external biological cues (EB)], motor imagery [internal biological cues (IB)], or geometrical figure sequences [external
nonbiological cues (EN)]. As hypothesized, we found that each condition elicited significant activation within PMv [left hemisphere,
Brodman Area (BA) 6], in contrast to a sequential target detection control task. In addition, cue-specific activations were identified in
areas that were only engaged for biologically (action) cued (EB, IB) and nonbiologically cued (EN) tasks. Biologically cued tasks elicited
activations within inferior frontal gyri adjacent to PMv (BA 44/45), in the frontomedian wall, the extrastriate body area, posterior
superior temporal sulci, somatosensory cortices, and the amygdala-hippocampal-area, whereas the nonbiologically cued task engaged
presupplementary motor area, middle frontal gyri, intraparietal sulci, and caudate nuclei of the basal ganglia. In sum, findings point to a
basic premotor contribution to the representation or processing of sequentially structured events, supplemented by different sets of areas
in the context of either biological or nonbiological cues.
Key words: premotor cortex; temporal order; perceptual prediction; action observation; biological motion; motor imagery; fMRI; se-
quencing; prospective attention
Introduction
The ability to anticipate a perceptual event on the basis of internal
or external cues speeds up our receptive processes and enables us
to react quickly and appropriately. It plays a fundamental role in
incredibly different domains such as motor control, during
which “internal forward models” turn motor commands into
expected sensory consequences (Jordan, 1996), and also in social
life and communication, during which saying “I know what he is
doing” expresses a conscious awareness of an agent’s future goal
(Gallese and Goldman, 1998); however, the neural constituents
contributing to anticipatory processes are still obscure.
Conceptually, the formation of expectations has to rely on the
representation of sequentially structured events, something like
“A precedes B.” One of the open questions that has not yet been
tested directly is whether sequential representations based on bi-
ological cues (observed or imagined actions) and those based on
nonbiological cues (abstract, sequentially ordered stimuli) re-
cruit overlapping brain networks, i.e., whether cue-independent
areas are engaged in sequential representation.
A candidate area is the ventral premotor cortex (PMv), which
is engaged by covert stages of action (Decety et al., 1994; Jean-
nerod and Decety, 1995; Hanakawa et al., 2003) and also by se-
quential tasks that operate on various nonbiological cues [for
meta-analysis and overview, see Schubotz and von Cramon
(2003)]. Animal data suggest that PMv houses an action reper-
toire with key constituents that are sensorimotor neurons, tuned
to the accomplishment of different action goals such as grasping
or reaching (Rizzolatti and Fadiga, 1998). Most interesting for the
present purpose, however, is that many of those PMv neurons
also show a temporal tuning, that is, they are specifically engaged
in parts of an action such as hand aperture or hand closure during
grasping (Rizzolatti et al., 1988). Temporal tuning, as a core ele-
ment of sequential representations inherent to movement and
action, manifests also in dorsal PM (Boussaoud et al., 1998)
and medial PM [supplementary motor area (SMA)] (Shima and
Tanji, 2000). Endowed with these particular neuronal properties
and given its robust involvement in nonbiologically cued sequen-
tial representations, the question arises whether PMv may be
exploited for both biological and nonbiological (abstract) se-
quential representations (Schubotz and von Cramon, 2004).
The present study investigated this hypothesis using func-
tional magnetic resonance imagining (fMRI). First, we aimed to
identify a PMv region that is activated both by observation of
biological motion [external biological cues (EB)] and by imag-
ined action [internal biological cues (IB)]. For both EB and IB, we
used a predictive instruction that allowed us to ensure and test
Received Feb. 11, 2004; revised May 3, 2004; accepted May 3, 2004.
We thank Sophie Manthey for the stimulus material, Andrea Gast-Sandmann for the figures, Ann-Shirley
Rueschemeyer for proof reading and suggestions, Uta Wolfensteller, Thomas Jacobsen, Evelyn Ferstl, Marcel Brass,
andChristian Fiebachfor helpfulcomments onthis manuscript,Gaby Lohmannand KarstenMu¨ller forsupport inMRI
statistics, and Katrin Sakreida for experimental assistance.
Correspondence should be addressed to Ricarda I. Schubotz, Department of Neurology, Max-Planck-Institute for
Human Cognitive and Brain Sciences, P.O. Box 500 355, 04103 Leipzig, Germany. E-mail schubotz@cbs.mpg.de.
DOI:10.1523/JNEUROSCI.1169-04.2004
Copyright © 2004 Society for Neuroscience 0270-6474/04/245467-08$15.00/0
The Journal of Neuroscience, June 16, 2004 •24(24):5467–5474 • 5467

on-line whether participants’attention was focused on the goal of
the currently observed or imagined action. If one assumes an area
within the PMv that contributes to representation of goal-
directed action, this area should be identified by this procedure.
Second, we tested whether the same PMv area is also engaged in
predictions that are cued by abstract nonbiological motion [ex-
ternal nonbiological cues (EN)], as implemented in a serial pre-
diction task (SPT) (Schubotz, 1999; Schubotz and von Cramon,
2003). Although biological and nonbiological cues were expected
to recruit additionally different areas, the representation of a
minimal sequence of invariant perceptual states was intended to
be shared by all experimental conditions. This hypothesis fol-
lowed the rationale of Byrne and Russon (1998), according to
which essential goal and subgoals of an action recur in every
effective action sequence, whereas the irrelevant details of pre-
cisely how each of these intermediate states is achieved will vary
between occasions without affecting efficiency. As such, goals and
subgoals are the predictable constituents of an observed action;
they are the most invariant perceptual states that an agent plans to
cause and that both the actor and a potential observer expect (and
hence predict) to perceive.
Materials and Methods
Participants. Eighteen right-handed, healthy young volunteers (10 fe-
male, 8 male; age range, 21–34 years; mean age 25.9 years) participated in
the study. After being informed about potential risks and screened by a
physician of the institution, subjects gave informed consent before par-
ticipating. The experimental standards were approved by the local ethics
committee of the University of Leipzig. Data were handled anonymously.
Stimuli. The paradigm comprised three experimental conditions (Fig.
1) and one baseline condition. Thirty-six trials were presented per con-
dition, and conditions were presented in random order (mixed-trial de-
sign). Within each trial of each condition, visual stimulation lasted 6 sec
and was preceded by a 2 sec verbal task cue; stimulation was followed by
the response phase and an intertrial interval that lasted 4 sec in total.
In condition EB, movies showing bimanual actions were presented
that were familiar from everyday life; for example, cutting a piece of
paper with scissors, lacing a shoe, shuffling cards, taking butter with a
knife, or opening a can. Movies always showed both hands and two
objects. In 50% of EB trials, the presented action resulted in an action slip
that rendered the final achievement of the intended action goal unlikely.
As confirmed by the participants’reports after the experiment, these
action slips were not detected immediately at the beginning of the movie
but required a careful observation until the end of the presentation.
Conversely, because participants were aware of action slips, they reported
that they observed carefully all EB trials, including those presenting cor-
rect actions, until the movie ended to make an appropriate choice.
In condition IB, movies were presented showing two hands holding
and moving one of two concurrently presented objects (the second, ir-
relevant object was presented to balance the number of objects presented
in EB and IB). In contrast to condition EB, however, the displayed hands
did not perform a goal-directed action but just “played around”with the
object. This motion was not task relevant in IB (see Task instructions
below) but was used only to match and balance the perceptual motion
information provided by each of the experimental conditions. The stim-
Figure 1. Experimental conditions. All trials followed the same temporal schema, depicted in the top panel. Examples for each condition (EB, IB, EN) are depicted in the bottom panels. In action
observation (EB), participants were asked to indicate whether the presented action was performed correctly; in motor imagery (IB), they were asked whether the object held and turned within the
actor’shandscould bemanually compressed; andin non-biologicalsequences(EN), theywere askedwhethera sequenceof abstract figuresfollowed aregularorder orif theorderwas violatedwithin
the last three stimuli of a trial. In the control condition (CO), participants were requested to indicate whether one frame contained a circle figure that differed from the concurrently presented ones.
Correct responses for the examples given in this figure were yes (observed action was performed correctly) (EB), no (manipulated object is not soft) (IB), yes (order of pictures was correctly repeated)
(EN), and yes (a deviant element was detected in one of the last pictures) (CO).
5468 •J. Neurosci., June 16, 2004 •24(24):5467–5474 Schubotz and von Cramon •Premotor Cortex in Abstract Sequences and Action

ulus material used in conditions EB and IB was also partly used in a
previous fMRI study (Manthey et al., 2003).
In condition EN, 12 different objects were used, each composed of a 25
mm circle (0.148 of visual angle) and a slightly smaller geometrical form
placed in its center. In six objects, this was a 14 mm square, and in the six
other objects it was a 10 mm circle. The big circle and the small form were
colored red, yellow, or blue, in such a manner that objects were always
two-colored. The same stimulus material was used in a previous study
(Schubotz and von Cramon, 2001). On each screen, an array of six identical
objects was presented. The size of the array was matched with that in condi-
tions EB and IB (⬍5°of visual angle). Within one EN trial, a sequence of 12
screens was presented subsequently for 500 msec each, with objects changing
from one screen to the next. The succession of the objects presented within
each trial exposed a regular order such that three objects built up a sequence
that was repeated four times. In 50% of the EN trials, the order of the last two
screens was flipped, resulting in a violation of the sequential object order. In
the control condition (CO), the same type of stimulus was presented as in the
EN condition, but the screen-to-screen order of objects was completely ir-
regular (random). In 50% of the CO trials, one of the last two screens was a
predefined target, containing one object of deviant color-form combination,
as compared with the other five concurrently presented stimuli.
Task instructions. For conditions EB and EN, participants were in-
structed to attend to the stimulus presentation and to predict how this
stimulation would evolve. In the EB condition, this instruction directed
the participants’attention to the intended action goal, whereas in the EN
condition, participants’attention was drawn to the sequential object
order. In both EB and EN, prediction was tested by a forced-choice
response after the end of the stimulus presentation. After action obser-
vation in EB, participants had to indicate whether the action was per-
formed in a goal-directed manner (button “yes”) or not (button “no”).
After abstract motion observation in EN, participants had to indicate
whether the object sequence was regular until the end of presentation
(button yes) or not (button no).
In condition IB, participants were asked to identify the displayed ob-
ject and judge whether it was a soft object, i.e., whether it would be
deformed when compressed with the hand (button yes) or not (button
no). By this instruction, participants were required to imagine them-
selves compressing the observed object and to make a judgment on the
basis of the imagined tactile–kinesthetic outcome of this action. It is
important to note that the turning of the object in the actor’s hands
provided no relevant information for the task itself, i.e., objects were not
compressed by the displayed hands, but was rather used to adapt the
amount of motion information provided by IB to that provided in EB
and EN.
After trials of condition CO, participants had to indicate whether the
trial contained the predefined target array (button yes) or not (button
no). Because the sequential order of stimuli was irrelevant for identifying
targets in the baseline condition, this condition did not require
prediction.
Data acquisition. Participants were instructed before the MRI experi-
ment. In the MRI session, subjects were supine on the scanner bed with
their right index and middle finger positioned on the response buttons.
To prevent postural adjustments, the subjects’arms and hands were
carefully stabilized by tape. In addition, form-fitting cushions were used
to prevent arm, hand, and head motion. Participants were provided with
earplugs to attenuate scanner noise.
Imaging was performed at 3T on a Bruker Medspec 30/100 system
equipped with the standard birdcage head coil. Sixteen axial slices (field
of view 192 mm; 64 ⫻64 pixel matrix; thickness 5 mm; spacing 2 mm)
parallel to bicommissural line (AC–PC) were acquired using a single-
shot gradient echo-planar imaging (EPI) sequence (echo time, 30 msec;
flip angle, 90°; repetition time, 2000 msec) sensitive to blood oxygenation
level-dependent contrast. A set of two-dimensional (2D) anatomical im-
ages was acquired for each subject immediately before the functional
experiment, using a modified driven equilibrium Fourier transformation
(MDEFT) sequence (256 ⫻256 pixel matrix). In a separate session,
high-resolution whole-brain images were acquired from each subject to
improve the localization of activation foci using a T1-weighted three-
dimensional (3D) segmented MDEFT sequence covering the whole
brain.
Data analysis. The MRI data were processed using the software pack-
age LIPSIA (Lohmann et al., 2001). Functional data were corrected for
motion using a matching metric based on linear correlation. To correct
for the temporal offset between the slices acquired in one scan, a sinc-
interpolation based on the Nyquist–Shannon theorem was applied. A
temporal high-pass filter with a cutoff frequency of 1/96 Hz was used for
baseline correction of the signal. No spatial filter was applied. To align the
functional data slices with a 3D stereotactic coordinate reference system,
a rigid linear registration with 6 df (3 rotational, 3 translational) was
performed. The rotational and translational parameters were acquired
on the basis of the MDEFT and EPI-T1 slices to achieve an optimal match
between these slices and the individual 3D reference data set. This 3D
reference data set was acquired for each subject during a previous scan-
ning session. The MDEFT volume data set with 160 slices and 1 mm slice
thickness was standardized to the Talairach stereotactic space (Talairach
and Tournoux, 1988). The rotational and translational parameters were
subsequently transformed by linear scaling to a standard size. The result-
ing parameters were then used to transform the functional slices using
trilinear interpolation, so that the resulting functional slices were aligned
with the stereotactic coordinate system. This linear normalization
process was improved by a subsequent processing step that performed
an additional nonlinear normalization (Thirion, 1998). Slice gaps
were interpolated to generate output data with a spatial resolution of
3⫻3⫻3 mm.
The statistical evaluation was based on a least-squares estimation using
the general linear model for serially autocorrelated observations (Friston,
1994; Friston et al., 1995a,b; Worsley and Friston, 1995). The design
matrix was generated with a synthetic hemodynamic response function
(Josephs et al., 1997; Friston et al., 1998) and its first and second deriva-
tive. To tap prediction-related processes, brain activations were analyzed
in an event-related design, time-locked to a point 2 sec after stimulus
onset. Only correctly answered trials entered the analysis. The model
equation, including the observation data, the design matrix, and the error
term, was convolved with a Gaussian kernel of dispersion of 4 sec full
width at half maximum to deal with the temporal autocorrelation (Wors-
ley and Friston, 1995). In the following, contrast images, i.e., estimates of
the raw-score differences between specified conditions, were generated
for each participant. The single-participant contrast images were then
entered into a second-level random effects analysis for each of the con-
trasts. The group analysis consisted of a one-sample ttest across the
contrast images of all participants that indicated whether observed dif-
ferences between conditions were significantly distinct from zero
(Holmes and Friston, 1998). Subsequently, tvalues were transformed
into zscores. To protect against false positive activations, only regions
with zscore ⬎3.09 ( p⬍0.001; uncorrected) and with a volume ⬎540
mm (20 measured voxels) were considered. All reported activations sur-
vived a threshold corresponding to p⬍0.05 (corrected for multiple
comparisons) at the cluster level.
Results
Behavioral performance
Performance was assessed by error rates. A repeated-measures
ANOVA with the two-level factor TASK (prediction, control)
indicated a significant main effect (F
(1,17)
⫽121.9; p⬍0.0001),
with an error rate of 14.1% for prediction and 1.0% for condition
CO. A repeated-measures ANOVA with the three-level factor TYPE
(EB, EN, IB) also indicated a significant main effect (F
(2,34)
⫽7.9;
p⬍0.006), with an error rate of 17.9% for EB, 14.8% for EN, and
9.7% for IB.
MRI data
Contrasted with CO, each condition EB, IB, and EN caused sig-
nificant and overlapping activation within the left PMv (Fig. 2).
The foci of these activations were almost identical for the predic-
tion conditions, located at x⫽⫺50, y⫽7, z⫽29 for EB, at x⫽
Schubotz and von Cramon •Premotor Cortex in Abstract Sequences and Action J. Neurosci., June 16, 2004 •24(24):5467–5474 • 5469

⫺47, y⫽4, z⫽32 for IB, and at x⫽⫺44, y⫽7, z⫽32 for EN
(Table 1, available at www.jneurosci.org). Although right PMv
activations also overlapped between all three conditions, a local
activation maximum within the right PMv was found only for
condition EN (x⫽45, y⫽7, z⫽26), whereas in conditions EB
and IB, right PMv activation rather extended from a more ante-
rior activation focus in adjacent areas BA 44/45. Condition EN
showed additional left PMv activation located inferiorly to the
common peak (x⫽⫺53, y⫽7, z⫽11). Moreover, each predic-
tion contrast activated both superior colliculi and a left temporal
region, corresponding to the motion area (MT) described in pre-
vious studies (Tootell et al., 1995).
Conditions using biological cues (EB and IB) shared a set of
activations that were not significantly activated for nonbiological
cues (EN). These included bilateral inferior frontal gyri within BA
44/45, left frontomedian wall (FMW) within the vicinity of BA
9m, bilateral extrastriate body area (EBA) in the temporo-
occipital cortex, a region within the posterior superior temporal
sulci (pSTS) located ⬃20 mm anterior to the EBA activation
maximum, the collateral sulci within the anterior fusiform gyri
corresponding to the fusiform face area (FFA), the amygdala–
hippocampal area (AHA) bilaterally, the left posterior insula
(pINS), and both postcentral gyri corresponding to primary–
secondary somatosensory cortex (SI, SII).
Finally, each condition also showed a set of regions to be specifi-
cally engaged, including the right posterior insula and the left cere-
bellar cortex close to the vermis, which were found only for contrast
IB–CO, and bilateral foci within posterior lateral orbitofrontal cor-
tex (BA 47), which were observed only for EB–CO. Finally, increased
levels of activation for EN–CO were found in the left pre-
supplementary motor area (pre-SMA), the middle frontal gyri, the
intraparietal sulci, bilateral heads of the caudate nuclei, and a ven-
Figure2. Braincorrelates of experimentaltasks. Group-averaged (n⫽18)statistical maps ofsignificantly activated areasfor prediction basedon IB, EB,and EN cues,as opposed tononpredictive
control condition (CO). Z-maps were thresholded at z⫽3.09 ( p⬍0.05 corrected). Top panel shows left (left) and right (right) sagittal sections through activations within ventral premotor cortex
andadjacentBA 44/45. Inbottom panels, activationis overlaid ontoa white mattersegmentation as ananatomical reference image.For each verticalcolumn(IB–CO, EB–CO,andEN–CO), displayed
slices cut through Talairach coordinates z⫽29 and x⫽⫺3 (top panel), x⫽⫺48 and x⫽48 (middle panel), and x⫽42 and z⫽⫺6 (bottom panel). Common activations within left PMv are
highlighted. For anatomical abbreviations see Table 1 (available at www.jneurosci.org).
5470 •J. Neurosci., June 16, 2004 •24(24):5467–5474 Schubotz and von Cramon •Premotor Cortex in Abstract Sequences and Action

tral–caudal portion of the fusiform gyri probably corresponding to
the lateral occipital complex (LOC). Figure 3 shows overlapping and
nonoverlapping activations of contrasts EB–CO, IB–CO, and
EN–CO superimposed on an individual brain.
Discussion
The PMv is suggested to play a role in covert stages of action,
including those triggered by action observation and action imag-
ery, but also in making predictions cued by nonbiological stimuli.
Pursuing the view that these different domains engage the PMv
because the representation of sequential events is common to all
of them, the present study tested whether a common PMv region
is activated by outcome-related tasks on action observation, ac-
tion imagery, or abstract sequences. Although correlates of bio-
logically and nonbiologically cued conditions clearly differed
within other brain areas, the overlapping activations within PMv
substantiate its contribution to sequential representations fed by
different cues.
Premotor overlap of EB and IB confirms a strong coupling
between observed and imagined actions in this cortical area, as
already suggested by corticospinal facilitation effects and event-
related brain potentials (Schnitzler et al., 1997; Hari et al., 1998;
Senkfor et al., 2002; Patuzzo et al., 2003; Clark et al., 2004). The
role of the PMv in this coupling has been specified as a matching
between stored and observed actions (Jeannerod, 2001), a spe-
cific application of the general matching functions in PMv. In
animal literature, the basis of perception-action matching has
been conceptualized as a “vocabulary of motor acts”stored
within the PMv (Rizzolatti et al., 1988; Rizzolatti and Fadiga,
1998). These vocabularies may be addressed either internally, i.e.,
by action planning, or by appropriate external stimuli, even in the
absence of motor intention (Murata et al., 1997). Initially neutral
with respect to their behavioral role (Fadiga et al., 2000), premo-
tor “vocabularies”can hence be exploited (1) for behavior and
imitation (production), when actions are generated from this
vocabulary (Rizzolatti and Gentilucci, 1988; Rizzolatti et al.,
1988; Jeannerod et al., 1995), and (2) for recognition and under-
standing (reception), when perceived objects or observed actions
are mapped onto it (Di Pellegrino et al., 1992; Rizzolatti et al.,
1996). Action goals rather than action means appear to be the
core of this tight perception-action linkage: the majority of F5
neurons discharge during the execution of goal-directed actions,
but not during comparable finger or hand movements (Rizzolatti
et al., 1988), and many F5 neurons code movements that share
the same goal but are performed with different effectors (Fadiga
et al., 2000). This goal preponderance can also be tapped
in human imitation (“theory of goal-directed imitation”)
(Wohlschla¨ger et al., 2003), and both imitation and other para-
digms focusing participants’attention on action goals were found
to activate human PMv (Decety et al., 1997; Grezes et al., 1999;
Chaminade et al., 2002, Johnson-Frey et al., 2003).
The present findings suggest that this apparently very flexible
“action vocabulary”in the PMv can also be exploited, at least in
humans, for much more abstract processes than assumed previ-
ously. Although abstract 2D color-form combinations used in
conditions EN (or more generally, in SPTs) do not have a one-
to-one motor implication, as real objects or observed actions
have, the PMv appears to be able to generate short-term tem-
plates of such stimulus sequences and therewith provide a basis
for perceptual predictions. Particularly in view of perceptual re-
sponses in monkey PMv, the acquisition and storage of abstract
perceptual templates in human PMv is conceivable in principle.
We have investigated this hypothesis in a series of fMRI studies,
particularly varying the perceptual information that has to be
predicted on the basis of regularly repetitive sequences. Condi-
tion EN confirmed that object prediction tasks, as opposed to
predictions tuned by spatial or rhythmic information, cause ac-
tivation within the PMv posteriorly, adjacent to the inferior fron-
tal sulcus (Schubotz and von Cramon, 2001, 2002a,b,c; Schubotz
et al., 2003). On the basis of the imaging literature (Schubotz and
von Cramon, 2003), this activation falls in a premotor area that
has been related to manual control and object attention. Using
observation of object manipulation in condition EB and imagery
of object manipulation in condition IB of the present study, we
expected that prediction on the basis of object-specific cues
(color and form) provided in EN would overlap with both EB and
IB effects in this part of the PMv because of a habitually consoli-
dated hand–object correspondence and hence object–property
preference in this area. The abstractness of representation impli-
cated here corresponds very nicely to the view that perceived
events and to-be-produced events (action goals) are represented
by composite codes of their “distal”features (Hommel et al.,
2002). Precursors of this theory implicate the (pre)motor cortices
in providing this coding interface (Liberman et al., 1967; Rizzo-
latti et al., 1987).
Beyond shared operations mediated by the PMv, activation in
other brain areas reflected specifically either biologically or non-
biologically cued conditions. Particularly of interest, EB and IB
activated BA 44/45 (including Broca’s area and its right ho-
molog). On the basis of the monkey literature, this activation may
reflect responses of the so-called “mirror neurons”(Gallese et al.,
1996; Rizzolatti et al., 1996), which appear to be endowed with
Figure 3. Overlap between experimental tasks. Significantly activated voxels of contrast
IB–CO (yellow), EB–CO (blue), and EN–CO (red) are displayed as an overlay map. White fields
show overlapping activations of all three conditions, whereas other colors indicate overlaps
between two or three conditions (green indicates IB and EB, purple indicates EB and EN, and
orange indicates EN and IB). The central sulcus is highlighted in white. The present study fo-
cusedonthe overlapofall conditions(whitefields), whichwasespecially prominentinthe PMv.
PMv activations are also indicated on brain schemas in the middle panel to focus on the ana-
tomical localization (sulci were redrawn from the reference brain). Further overlap between all
conditionswaslocated inthe left motionarea andthesuperior colliculi.Overlap betweenEBand
IB was found in the fusiform face area, extrastriate body area, and inferior frontal gyri.
Schubotz and von Cramon •Premotor Cortex in Abstract Sequences and Action J. Neurosci., June 16, 2004 •24(24):5467–5474 • 5471

the properties that simulation theories require (Gallese and
Goldman, 1998; Carr et al., 2003; Miall, 2003). Hence, the tem-
plate matching mediated by PMv–BA6 might have been supple-
mented, only in the case of biological cues, by goal representa-
tions in anteriorly adjacent PMv–BA 44 (Umilta` et al., 2001).
According to this view, condition EB required one specific in-
stance of an action (as shown in the movie) to be matched with a
standard template of that action (invariant features), probably
stored in BA 44 (Rizzolatti et al., 2001). Likewise, imageries re-
quired in IB were conceivably generated from such a standard
template; however, because Broca’s area is also known to mediate
linguistic representations, especially in tasks that require strategic
matching (Bookheimer, 2002), an alternative interpretation
could come from ontogenetic and phylogenetic accounts, stress-
ing the general role of Broca’s area in hierarchically organized
sequential behavior (Greenfield, 1991). Similar to the role of pre-
SMA in the organization of abstract sequences (Shima and Tanji,
2000), BA 44 may contribute to the generation of higher-level
organization in actions, running on lower-level, serially orga-
nized representations within adjacent PMv–BA6.
The remaining discussion will focus on additional areas spe-
cifically engaged for EB and IB, because areas exclusively acti-
vated by EN are a direct replication of previous findings and have
been discussed previously (Schubotz and von Cramon, 2003).
Moreover, some of these areas cannot be discussed in detail: SI/
SII and pINS, which can be taken to reflect tactile–kinesthetic
imagery (Carlsson et al., 2000; Bushara et al., 2001; Yoo et al.,
2003), and middle cerebellar cortex activity in IB, which we take
to confirm the cerebellar role in predicting the specific sensory
consequences of movements (Blakemore et al., 2001).
The FMW (BA 9m) is reciprocally connected with lateral PM
(Barbas et al., 1999). Imaging studies relate activation within BA
9m to (1) the ability to distinguish between actions executed by
oneself and actions executed by others (Brass et al., 2001; Ruby
and Decety, 2001) and (2) self-referential representations (Ferstl
and von Cramon, 2002; Zysset et al., 2002) that help to distin-
guish between our own and another person’s perspective (Frith
and Frith, 1999; Gallagher et al., 2002; Gallagher and Frith, 2003).
This interpretation fits well with our findings, because EB re-
quired participants to take the perspective of the presented actor
and subsequently to distinguish carefully between the actor’s per-
formance and what the participant himself or herself would do to
achieve the inferred goal. Although condition IB did not require
such a perspective, participants had to identify an aimlessly ma-
nipulated object. We suggest that seeing this movement inter-
fered with the requirement to imagine oneself pressing the object
to determine its softness, i.e., a highly similar movement (Brass et
al., 2001).
Posterior STS and bilateral EBA were activated by EB and IB.
Although both areas are known to respond to biological stimuli,
only EBA is contingent on the shape of a body being explicitly
represented (Downing et al., 2001; Grossman and Blake, 2002),
whereas pSTS is rather driven by dynamics of biological stimuli
such as actions and gestures. Its putative homolog in macaques
(anterior superior temporal sulcus) is known to respond to the
observation of biological actions, just as F5c (Rizzolatti et al.,
2001), and contributes to the recognition and understanding of
the actions of other (Emery and Perrett 1994; Jellema et al., 2002),
an interpretation that is paralleled in the human literature (for
review, see Allison et al., 2000). The present pSTS coordinates
resemble those reported especially for perception of hand (as
compared with body) motion (Bonda et al., 1996). Posterior STS
activation in conditions EB and IB corresponded nicely to acti-
vation within two additional areas that are known to be con-
nected with pSTS (Barbas, 1988; Amaral et al., 1992), the poste-
rior lateral orbitofrontal cortex and the AHA. On the basis of a
review on orbitofrontal cortex and amygdala (Adolphs, 1999), we
take their activation in IB and EB to reflect the mediation between
perceptual representations of the sight of conspecifics and re-
trieval of knowledge triggered by such stimuli. Finally, EB and IB
caused specific activation within the FFA (Kanwisher et al., 1997),
which is known to respond not only to faces but also to highly
familiar objects (Gauthier et al., 1999). In contrast, the caudal–
ventral sector of the LOC activated by condition EN responds
more generally to object shapes (Grill-Spector et al., 2001). In
sum, occipital and temporal activations consistently reflected
specific requirements in the context of the interpretation of bio-
logical as opposed to nonbiological cues.
Conclusion
Recent research has discarded the classical “motor planning”
concept of premotor cortex in favor of a multipurpose action–
perception matching interface. The present findings suggest that
this interface can be exploited for the representation of sequen-
tially structured events in a broader range of behaviors as as-
sumed previously. Although each implementation asks for addi-
tional contributions from a set of specialized areas, tasks that rely
on sequentially structured information may always converge on
the PMv, be it in the context of planned, imagined, executed, or
observed actions or in spatiotemporal trajectories or abstract se-
quences defined by numerical, figural, or acoustic stimuli. Even
in the absence of biologically or pragmatically meaningful stim-
uli, instructed prediction suffices to activate the PMv; however,
that is not to say that only prediction engages PMv. Future studies
must try to settle which common operations provided by the
PMv are shared in biologically and nonbiologically cued nonmo-
tor tasks.
References
Adolphs R (1999) Social cognition and the human brain. Trends Cogn Sci
3:469–479.
Allison T, Puce A, McCarthy G (2000) Social perception from visual cues:
role of the STS region. Trends Cogn Sci 4:267–278.
Amaral DG, Price JL, Pitkaenen A, Carmichael ST (1992) Anatomical orga-
nization of the primate amygdaloid complex. In: The amygdala: neurobi-
ological aspects of emotion, memory, and mental dysfunction (Aggleton
JP, ed), pp 1–66. New York: Wiley.
Barbas H (1988) Anatomic organization of basoventral and mediodorsal
visual recipient prefrontal regions in the rhesus monkey. J Comp Neurol
276:313–342.
Barbas H, Ghashghaei H, Dombrowski SM, Rempel-Clower NL (1999) Me-
dial prefrontal cortices are unified by common connections with superior
temporal cortices and distinguished by input from memory-related areas
in the rhesus monkey. J Comp Neurol 410:343–367.
Blakemore SJ, Frith CD, Wolpert DM (2001) The cerebellum is involved in
predicting the sensory consequences of action. NeuroReport
12:1879–1884.
Bonda E, Petrides M, Ostry D, Evans A (1996) Specific involvement of hu-
man parietal systems and the amygdala in the perception of biological
motion. J Neurosci 16:3737–3744.
Bookheimer S (2002) Functional MRI of language: new approaches to un-
derstanding the cortical organization of semantic processing. Annu Rev
Neurosci 25:151–188.
Boussaoud D, Jouffrais C, Bremmer F (1998) Eye position effects on the
neuronal activity of dorsal premotor cortex in the macaque monkey.
J Neurophysiol 80:1132–1150.
Brass M, Zysset S, von Cramon DY (2001) The inhibition of imitative re-
sponse tendencies. NeuroImage 14:1416–1423.
Bushara KO, Wheat JM, Khan A, Mock BJ, Turski PA, Sorenson J, Brooks BR
5472 •J. Neurosci., June 16, 2004 •24(24):5467–5474 Schubotz and von Cramon •Premotor Cortex in Abstract Sequences and Action

(2001) Multiple tactile maps in the human cerebellum. NeuroReport
12:2483–2486.
Byrne RW, Russon AE (1998) Learning by imitation: a hierarchical ap-
proach. Behav Brain Sci 21:667–684.
Carlsson K, Petrovic P, Skare S, Petersson KM, Ingvar M (2000) Tickling
expectations: neural processing in anticipation of a sensory stimulus. J
Cognit Neurosci 12:691–703.
Carr L, Iacoboni M, Dubeau MC, Mazziotta JC, Lenzi GL (2003) Neural
mechanisms of empathy in humans: a relay from neural systems for imi-
tation to limbic areas. Proc Natl Acad Sci USA 100:5497–5502.
Chaminade T, Meltzoff AN, Decety J (2002) Does the end justify the means?
A PET exploration of the mechanisms involved in human imitation. Neu-
roImage 15:318–328.
Clark S, Tremblay F, Ste-Marie D (2004) Differential modulation of corti-
cospinal excitability during observation, mental imagery and imitation of
hand actions. Neuropsychologia 42:105–112.
Decety J, Perani D, Jeannerod M, Bettinardi V, Tadary B, Woods R, Mazziotta
JC, Fazio F (1994) Mapping motor representations with positron emis-
sion tomography. Nature 371:600–602.
Decety J, Grezes J, Costes N, Perani D, Jeannerod M, Procyk E, Grassi F, Fazio
F (1997) Brain activity during observation of actions. Influence of action
content and subject’s strategy. Brain 120:1763–1777.
Di Pellegrino G, Fadiga L, Fogassi L, Gallese V, Rizzolatti G (1992) Under-
standing motor events: a neurophysiological study. Exp Brain Res
91:176–180.
Downing PE, Jiang Y, Shuman M, Kanwisher N (2001) A cortical area selec-
tive for visual processing of the human body. Science 293:2470–2473.
Emery NJ, Perrett DI (1994) Understanding the intentions of others from
visual signals: neurophysiological evidence. Curr Psychol Cogn
13:683–694.
Fadiga L, Fogassi L, Gallese V, Rizzolatti G (2000) Visuomotor neurons:
ambiguity of the discharge or “motor”perception? Int J Psychophysiol
35:165–177.
Ferstl EC, von Cramon DY (2002) What does the frontomedian cortex con-
tribute to language processing: coherence or theory of mind? NeuroImage
17:1599–1612.
Friston KJ (1994) Statistical parametric mapping. In: Functional neuroim-
aging (Thatcher RW, Hallet M, Zeffiro T, John ER, Huerta M, eds), pp
79–93. San Diego: Academic.
Friston KJ, Holmes AP, Poline JB, Grasby BJ, Williams CR, Frackowiak RSJ,
Turner R (1995a) Analysis of MRI time-series revisited. NeuroImage
2:45–53.
Friston KJ, Holmes AP, Worsley KJ, Poline JP, Frith CD, Frackowiak RSJ
(1995b) Statistical parametric maps in functional imaging: a general lin-
ear approach. Hum Brain Mapp 2:189–210.
Friston KJ, Fletcher P, Josephs O, Holmes A, Rugg MD, Turner R (1998)
Event-related fMRI: characterizing differential responses. NeuroImage
7:30–40.
Frith CD, Frith U (1999) Interacting minds—a biological basis. Science
286:1692–1695.
Gallagher HL, Frith CD (2003) Functional imaging of “theory of mind”.
Trends Cogn Sci 7:77–83.
Gallagher HL, Jack AI, Roepstorff A, Frith CD (2002) Imaging the inten-
tional stance in a competitive game. NeuroImage 16:814–821.
Gallese V, Goldman A (1998) Mirror neurons and the simulation theory of
mind-reading. Trends Cogn Sci 2:493–501.
Gallese V, Fadiga L, Fogassi L, Rizzolatti G (1996) Action recognition in the
premotor cortex. Brain 119:593–609.
Gauthier I, Tarr MJ, Anderson AW, Skudlarski P, Gore JC (1999) Activation
of the middle fusiform “face area”increases with expertise in recognizing
novel objects. Nat Neurosci 2:568–573.
Greenfield PM (1991) Language, tools, and brain: the ontogeny and phylog-
eny of hierarchically organized sequential behavior. Behav Brain Sci
14:531–551.
Grezes J, Costes N, Decety J (1999) The effects of learning and intention on
the neural network involved in the perception of meaningless actions.
Brain 122:1875–1887.
Grill-Spector K, Kourtzi Z, Kanwisher N (2001) The lateral occipital com-
plex and its role in object recognition. Vision Res 41:1409–1422.
Grossman ED, Blake R (2002) Brain areas active during visual perception of
biological motion. Neuron 35:1167–1175.
Hanakawa T, Immisch I, Toma K, Dimyan MA, Van Gelderen P, Hallett M
(2003) Functional properties of brain areas associated with motor exe-
cution and imagery. J Neurophysiol 89:989–1002.
Hari R, Forss N, Avikainen S, Kirveskari E, Salenius S, Rizzolatti G (1998)
Activation of human primary motor cortex during action observation: a
neuromagnetic study. Proc Natl Acad Sci USA 95:15061–15065.
Holmes AP, Friston KJ (1998) Generalisability, random effects and popula-
tion inference. NeuroImage 7:S754.
Hommel B, Mu¨sseler J, Aschersleben G, Prinz W (2002) The theory of event
coding (TEC): a framework for perception and action planning. Behav
Brain Sci 24:849–878.
Jeannerod M (2001) Neural simulation of action: a unifying mechanism for
motor cognition. NeuroImage 14:103–109.
Jeannerod M, Decety J (1995) Mental motor imagery: a window into the
representational stages of action. Curr Opin Neurobiol 5:727–732.
Jeannerod M, Arbib MA, Rizzolatti G, Sakata H (1995) Grasping objects:
the cortical mechanisms of visuomotor transformation. Trends Neurosci
18:314–320.
Jellema T, Baker CI, Oram MW, Perrett DI (2002) Cell populations in the
banks of the superior temporal sulcus of the macaque and imitation. In:
The imitative mind (Prinz W, Meltzoff A, eds), pp 267–290. Cambridge,
UK: Cambridge UP.
Johnson-Frey SH, Maloof FR, Newman-Norlund R, Farrer C, Inati S, Grafton
ST (2003) Actions or hand-object interactions? Human inferior frontal
cortex and action observation. Neuron 39:1053–1058.
Jordan MI (1996) Computational aspects of motor control and motor
learning. In: Handbook of perception and action: motor skills (Heuer H,
Keele S, eds), pp 71–118. New York: Academic.
Josephs O, Turner R, Friston K (1997) Event-related fMRI. Hum Brain
Mapp 5:243–248.
Kanwisher N, McDermott J, Chun MM (1997) The fusiform face area: a
module in human extrastriate cortex specialized for face perception.
J Neurosci 17:4302–4311.
Liberman AM, Cooper FS, Shankweiler DP, Studdert-Kennedy M (1967)
Perception of the speech code. Psychol Rev 74:431–461.
Lohmann G, Mueller K, Bosch V, Mentzel H, Hessler S, Chen L, Zysset S, von
Cramon DY (2001) Lipsia—a new software system for the evaluation of
functional magnetic resonance images of the human brain. Comp Med
Imag Graph 25:449–457.
Manthey S, Schubotz RI, von Cramon DY (2003) Premotor cortex in ob-
serving erroneous action: an fMRI study. Cognit Brain Res 15:296–307.
Miall RC (2003) Connecting mirror neurons and forward models. Neuro-
Report 14:2135–2137.
Murata A, Fadiga L, Fogassi L, Gallese V, Raos V, Rizzolatti G (1997) Object
representation in the ventral premotor cortex (area F5) of the monkey.
J Neurophysiol 78:2226–2230.
Patuzzo S, Fiaschi A, Manganotti P (2003) Modulation of motor cortex ex-
citability in the left hemisphere during action observation: a single- and
paired-pulse transcranial magnetic stimulation study of self- and non-
self-action observation. Neuropsychologia 41:1272–1278.
Rizzolatti G, Fadiga L (1998) Grasping objects and grasping action mean-
ings: the dual role of monkey rostroventral premotor cortex (area F5).
Novartis Found Symp 218:81–103.
Rizzolatti G, Gentilucci M (1988) Motor and visual-motor functions of the
premotor cortex. In: Neurobiology of neocortex (Rakic P, Singer W, eds),
pp 269–284. Chichester, UK: Wiley.
Rizzolatti G, Riggio L, Dascola I, Umilta C (1987) Reorienting attention
across the horizontal and vertical meridians: evidence in favor of a pre-
motor theory of attention. Neuropsychologia 25:31–46.
Rizzolatti G, Camarda R, Fogassi L, Gentilucci M, Luppino G, Matelli M
(1988) Functional organization of inferior area 5 in the macaque mon-
key. II: Area F5 and the control of distal movements. Exp Brain Res
71:491–507.
Rizzolatti G, Fadiga L, Gallese V, Fogassi L (1996) Premotor cortex and the
recognition of motor actions. Cognit Brain Res 3:131–141.
Rizzolatti G, Fogassi L, Gallese V (2001) Neurophysiological mechanisms
underlying the understanding and imitation of action. Nat Rev Neurosci
2:661–670.
Ruby P, Decety J (2001) Effect of subjective perspective taking during sim-
ulation of action: a PET investigation of agency. Nat Neurosci 4:546 –550.
Schnitzler A, Salenius S, Salmelin R, Jousmaki V, Hari R (1997) Involve-
ment of primary motor cortex in motor imagery: a neuromagnetic study.
NeuroImage 6:201–208.
Schubotz and von Cramon •Premotor Cortex in Abstract Sequences and Action J. Neurosci., June 16, 2004 •24(24):5467–5474 • 5473

Schubotz R (1999) Instruction differentiates the processing of temporal and
spatial sequential patterns: evidence from slow wave activity in humans.
Neurosci Lett 265:1–4.
Schubotz RI, von Cramon DY (2001) Functional organization of the lateral
premotor cortex: fMRI reveals different regions activated by anticipation
of object properties, location and speed. Cognit Brain Res 11:97–112.
Schubotz RI, von Cramon DY (2002a) Dynamic patterns make the premo-
tor cortex interested in objects: influence of stimulus and task revealed by
fMRI. Cognit Brain Res 14:357–369.
Schubotz RI, von Cramon DY (2002b) Predicting perceptual events acti-
vates corresponding motor schemes in lateral premotor cortex: an fMRI
study. NeuroImage 15:787–796.
Schubotz RI, von Cramon DY (2002c) A blueprint for target motion: fMRI
reveals perceptual complexity to modulate a premotor-parietal network.
NeuroImage 16:920–935.
Schubotz RI, von Cramon DY (2003) Functional-anatomical concepts of
human premotor cortex: evidence from fMRI and PET studies. Neuro-
Image 20 [Suppl 1]:120–131.
Schubotz RI, von Cramon DY (2004) Brains have emulators with brains:
emulation economized. Behav Brain Sci, in press.
Schubotz RI, von Cramon DY, Lohmann G (2003) Auditory what, where,
and when: a sensory somatotopy in lateral premotor cortex. NeuroImage
20:173–185.
Senkfor AJ, Van Petten C, Kutas M (2002) Episodic action memory for real
objects: an ERP investigation with perform, watch, and imagine action
encoding tasks versus a non-action encoding task. J Cognit Neurosci
14:402–419.
Shima K, Tanji J (2000) Neuronal activity in the supplementary and pre-
supplementary motor areas for temporal organization of multiple move-
ments. J Neurophysiol 84:2148–2160.
Talairach J, Tournoux P (1988) Co-planar stereotaxic atlas of the human
brain. New York: Thieme.
Thirion JP (1998) Image matching as a diffusion process: an analogy with
Maxwell’s demons. Med Image Anal 2:243–260.
Tootell RB, Reppas JB, Kwong KK, Malach R, Born RT, Brady TJ, Rosen BR,
Belliveau JW (1995) Functional analysis of human MT and related vi-
sual cortical areas using magnetic resonance imaging. J Neurosci
15:3215–3230.
Umilta` MA, Kohler E, Gallese V, Fogassi L, Fadiga L, Keysers C, Rizzolatti G
(2001) I know what you are doing: a neurophysiological study. Neuron
31:155–165.
Wohlschla¨ger A, Gattis M, Bekkering H (2003) Action generation and ac-
tion perception in imitation: an instance of the ideomotor principle. Phi-
los Trans R Soc Lond B Biol Sci 358:501–515.
Worsley KJ, Friston KJ (1995) Analysis of fMRI time-series revisited—
again. NeuroImage 2:173–181.
Yoo SS, Freeman DK, McCarthy JJ 3rd, Jolesz FA (2003) Neural substrates
of tactile imagery: a functional MRI study. NeuroReport 14:581–585.
Zysset S, Huber O, Ferstl E, von Cramon DY (2002) The anterior frontome-
dian cortex and evaluative judgment: an fMRI study. NeuroImage 15:
983–991.
5474 •J. Neurosci., June 16, 2004 •24(24):5467–5474 Schubotz and von Cramon •Premotor Cortex in Abstract Sequences and Action