Content uploaded by Catherine Joanna Stevens
Author content
All content in this area was uploaded by Catherine Joanna Stevens
Content may be subject to copyright.
The Effect of Implied Harmony, Contour and Musical Expertise
on Judgments of Similarity of Familiar Melodies
Emery Schubert
1
and Catherine Stevens
2
1
School of Music and Music Education, University of New South Wales, Australia;
2
School of Psychology & MARCS
Auditory Laboratories, University of Western Sydney, Australia
Abstract
According to Western music theory, familiar melodies
containing alterations which shift them a long distance
(harmonically) from the original should be considered
perceptually dissimilar relative to the original. It is
possible to obtain such a large shift with a small change
in melodic pitch. However, this creates a paradox with
pitch (contour) shifting, which is known to reduce
similarity minimally if the distance from the original is
small. We investigated the hypotheses that (1) manip-
ulations to contour and implied harmony of an original
melody reduce similarity scores and (2) novices are less
sensitive to implied harmony changes than experts, but
as sensitive as experts to contour manipulations.
Twenty-eight novices and 44 expert musicians rated
similarity of familiar nursery rhyme tunes compared
with close and distant harmonic transformations plus
close and distant pitch shifts. Results indicated that
both groups use contour (pitch distance) to determine
similarity, but that musically experienced listeners also
use implied harmony to make further distinctions. It is
argued that as listeners become more experienced, they
rely on more sophisticated strategies for encoding and
organizing melodies in memory, with deeper structural
aspects of music being used when other strategies such
as contour are uninformative or non-distinctive. These
new findings, that contour and harmony affect judged
similarity for simple, familiar melodies, have implica-
tions for theories of memory for music, and for the
design of automated music retrieval and data mining
systems.
1. Introduction
The study of perceived melodic similarity provides
psychologists with information about how the brain
processes, stores and retrieves musical information.
Studies of perceived melodic similarity are also impor-
tant for search and retrieval systems (Kim et al. 2000;
Byrd & Crawford, 2002; Zhu & Kankanhalli, 2002),
compression algorithms (e.g. Hofmann-Engl, 2001), and
to further our understanding of processes involved in
perceiving ‘‘theme and variations’’ and recognizing music
in naturalistic listening settings (Deliege, 2001; Lamont &
Dibben, 2001; McAdams & Matzkin, 2001).
If a presentation of Melody A is rated as being similar
to, or confused with (Dowling & Bartlett, 1981), a
different Melody B then we can assume that the features
that differ across the two melodies have been assimilated
into the same melodic schemata activated in long-term
memory. One melody contains some kind of perceptual
redundancy with respect to the other melody. In other
words, the study of melodic similarity asks how
physically different a melody must be before it no longer
bears perceptual equivalence to its original form (e.g. see
McAdams & Matzkin, 2001). However, the study of the
phenomenon is complicated by factors such as individual
differences, and the selection and definition of the
independent variables. Regarding the former, can we
assume that all listeners will judge two melodies as being
dissimilar by roughly the same amount? Regarding the
latter, should surface structures be investigated, such as
pitch and rhythm variations, or deeper structure, such as
the harmony implied by a melodic progression or
Correspondence: Emery Schubert, School of Music and Music Education, University of New South Wales, Sydney, 2052 NSW,
Australia. E-mail: E.Schubert@unsw.edu.au
Journal of New Music Research
2006, Vol. 35, No. 2, pp. 161 – 174
DOI: 10.1080/09298210600835000 Ó2006 Taylor & Francis
structural landmarks (for a discussion and definition of
surface and deep structure, see Lamont & Dibben, 2001)?
This paper examines some of these issues, and identifies
and investigates an apparent paradoxical effect of
harmony and contour upon judgment of perceptual
similarity.
2. Individual differences and melody familiarity
Judgement of melodic similarity is neither objective nor
universal. Many factors come in to play when we ask
‘‘How similar is Melody A to Melody B?’’. One factor is
the listener’s familiarity with the melodic material
(Welker 1982; Halpern, 1984a,b, 1989; Jones et al.
1987; McAdams & Matzkin, 2001). Dowling (1988)
found that children younger than about 3 years retrieve
basic melodic contour information from long-term
memory, but that older children use more specific
pitch-relationship information, facilitating more accurate
retrieval of the melodies from long-term memory (see
also Trehub et al. 1984; Trehub 1990, 2001; Dowling,
1994). However, even in adulthood, contour is still an
important component in cognitive organization of
melodic pitch streams (Dowling, 1994; Levitin, 1999;
Trainor et al. 1999, 2002).
Perceived differences in melodies may also be a
function of listeners’ aural acuity and experience. It is
common for experimental studies to exercise some
control over this participant factor. The most common
grouping is ‘‘musician’’ versus ‘‘non-musician’’. Yet
studies generally report little or unsurprising differences
between these different groups of people. In general, the
differences might be traced back to familiarity with
materials, a likely feature of the ‘‘musician’’ group.
Greater familiarity with a tune or the rules of the tune’s
genre should allow greater amounts of information to be
stored in memory and the use of more sophisticated
problem solving strategies (Anderson & Schooler, 2000;
Ericsson, 2003), facilitating accurate judgment. Perhaps
experienced musicians have a better understanding of
deep level musical structure than less experienced
musicians. Much of the literature examines responses
to surface level manipulations (such as pitch, contour
and rhythm). It is relevant, therefore, to investigate and
compare differences between the processing of so-called
deeper structures by musicians and non-musicians
(Lamont & Dibben, 2001).
3. The effect of musical features on perceived
similarity
What musical parameter manipulations lead to perceived
dissimilarity? Cambouropoulos (2000) notes that musical
context can affect judgments of the similarity of auditory
patterns. Levitin (1999) proposed that transformations in
timbre, pitch, loudness, spatial location, starting pitch
(or transposition – Van Egmond & Povel, 1996),
reverberation and tempo across a melodic transforma-
tion, in general, do not affect the identity of a melody. He
argues that rhythmic changes can have an effect but are
generally not strong. However, the pattern of ups and
downs in pitch have a significant effect on judged
similarity. The related issues of melodic contour and
implied harmony are the main focus of the present
investigation of melodic similarity.
3.1 Melodic contour
A number of definitions for melodic contour exist in the
literature. Edworthy (1985) summarizes them as
. . . ranging from a global description of the predominant
configuration of a musical theme . . . to a specific, note-by-
note representation of a melodic sequence taking into
account both interval size and direction of melodic move-
ment between adjacent notes. A midpoint between these
levels of analysis exists whereby contour is defined as a
sequence of ups and downs in a melody or tone sequence,
independent of precise interval size (pp. 375 – 376).
Edworthy’s midpoint definition is typical of those
reported in subsequent literature. However describing
contour as ‘‘patterns of ups and downs’’ can be
misleading because a contour is a line representing a
body or figure – the information about the outline of the
pattern is retained (see Schoenberg, 1967 for examples of
this detailed kind of contour). In contrast, the commonly
used definition of contour described by Edworthy is
information reduced. Patterns of ups and downs capture
melodic direction, and not the finer detail of the actual
contour drawn out by the pattern of pitches over time
(e.g. see Welker 1982). In fact, when comparing a melody
with respect to another, the ‘‘keep the contour same’’
method produces no change in the comparison melody,
making melodic direction have only two codable values
for testing – up and down. This is different to melodic
displacement, where the distance of the notes (chroma)
from the source melody is what becomes coded (for
example, in units of semitones). An increased movement
away from the original source note leads to an increase in
displacement, regardless of direction. Therefore, we
propose that what has been referred to in some literature
as melodic contour is, more accurately, melodic direction,
and that definitions which embrace multilevel positioning
away from a reference point refer to melodic displace-
ment (or distance)
1
.
1
In this paper we therefore use the term ‘‘pitch displacement’’ as
a subset of ‘‘contour’’.
162 Emery Schubert and Catherine Stevens
Kim et al. (2002) discuss various coding resolutions
comparing simple directional coding (up, down, same)
with five and seven level representations. Seven level
displacement allows a higher degree of resolution to be
coded in a test melody with respect to the original, as
shown in Figure 1. In practice only 6 levels are relevant in a
comparison task because the same/no-change condition is
the original source melody against which the comparison is
made (as shown in the horizontal centre of the figure). In
the figure there are three levels of displacement coded in the
test melody (‘‘3’’ being a large displacement, ‘‘1’’ being
small), and the sign indicates the direction of the change
(‘‘7’’ for changes in opposite direction with respect to
original note-to-note transitions, ‘‘þ’’ for changes in the
same direction with respect to original note-to-note
transitions). The Kim et al. study was an examination of
the physical differences and algorithmic efficiency of the
different methods and concluded that five level contour
produced an optimal compromise between the efficiency of
three level coding, and the robustness of seven level coding.
In the present study the coding of contour is an
important issue. It is possible to manipulate implied
harmony (discussed below) such that its theoretical effect
upon similarity is in opposition to the effect of contour.
For example, consider the situation where a test melody is
compared with a melody stored in memory. Contour
changes using the displacement definition are known to
degrade perceived similarity. This can be represented
visually as shown in Figure 1. With no change in physical
contour, a high similarity rating is provided. However, as
the melodic pitch shape is changed, similarity scores
reduce. When the melody is changed such that the size of
melodic steps and leaps are increased with respect to the
original, the similarity judgements decline, as shown in the
right half of Figure 1. When the direction of a step or leap
is changed, and the amount of this change is increased, this
too is associated with a decrease in perceived similarity, as
shown in the left half of Figure 1. This approach allows a
fairly simple means of quantifying change because the
difference (in semitones) of each note of a test melody
(compared with the corresponding note of the original) can
be calculated. These differences are then converted into a
single standard deviation score that quantifies how
different the contour of the test melody is from the original.
3.2 Implied harmony
To a Western listener, a melody that conforms to the
conventions of Western tonal music is more than a
collection of pitches and note durations. For example,
Bigand (1993) points out that the notation represents ‘‘a
complex network of tension and relaxation’’ (p. 51) and
Povel and Jansen argue that ‘‘harmonic relations
between consecutive tones play an important role in the
perception of music’’ (Jansen & Povel, 2004b, p. 47).
Dowling’s studies of contour (e.g. Dowling, 1978, 1991;
Dowling & Bartlett, 1981; Bartlett & Dowling, 1988)
suggest that scale structure is an important aspect of
melodic organization, and this has been supported in
studies by Povel and Jansen (Povel & Jansen, 2002;
Jansen & Povel, 2004a, 2004b). Alterations to a melody
that maintain the scale structure are considered more
similar than those in which scale structure is violated.
For example, if a piece is in C major, changing each g to
ag
#violates the expectation of diatonic (within-key)
notes being used. This produces lower ‘‘liking’’ scores
Fig. 1. Theoretical representation of the effect of contour (melodic displacement) variation upon similarity judgement. As the average
of consecutive pitch distance increases with respect to the original melody, similarity decreases, whether the changes are in the same
direction as the original transition (right side) or the opposite direction (left side).
Implied harmony, contour and expertise 163
(Cross et al. 1983), lower goodness ratings (Krumhansl &
Keil, 1982; Povel & Jansen, 2002), sounds more ‘‘funny’’
(Dowling, 1988), and is more noticeable (Dowling, 1978).
Bartlett & Dowling (1988) proposed several explanations
for why this deviation from the diatonic scale produced
such results. For the purpose of studying similarity,
deviation from the diatonic may present only a general
explanation of why foreign notes disrupt a melody.
According to this paradigm, any non-diatonic (out-of-key
or chromatic) transformation should affect similarity.
However, what would happen if a diatonic note was
substituted for the original – such as an ain place of a g
(assuming we are in the key of C major, to which both g
and abelong), instead of a g#? The story then becomes
more complex. Western music theory allows us to
determine whether the ais harmonically related to the
original note (e.g. see Krumhansl, 1990). More specifi-
cally, non-diatonic (chromatic) pitch substitutions in a
melody are less likely to fulfill the requirements of the
harmony implied by the melody in question. If this were
the case perceptually, it would provide evidence that
melodic information, particularly of veridical or stylistic
familiarity, is stored not just as a collection of contours,
pitches and rhythms, but also as a web of harmonic
implications. That is, harmonic structure is abstracted by
the listener. Such a notion is supported in experiments by
Tan et al. (1981) and Bharucha’s neural network model of
music perception (see Bharucha and Krumhansl, 1983;
Tillmann et al. 2000) and several more recent studies
(Tillmann et al. 2000; Povel & Jansen, 2002; Jansen &
Povel, 2004a; Tillmann & Bigand, 2004).
Research on perceived melodic similarity has focused
mainly on the parameters of contour or rhythm or both
(e.g. Davidson, 1985; Edworthy, 1985; Carterette et al.
1986; Jones et al. 1987; He
´bert & Peretz, 1997; Halpern
et al. 1998; Cambouropoulos & Widmer, 2000; Byrd &
Crawford, 2002). In contrast, implied harmony has
received relatively little explicit attention in studies of
melodic similarity (important exceptions are Sloboda &
Parker, 1985; Stoll & Parncutt, 1987; Trainor & Trehub,
1994b; Holleran et al. 1995). It is well understood by
music theorists that Western melodies can be harmonized
so as to support the melody (i.e. make it sound ‘‘good’’).
Further, there are a limited number of solutions to the
harmonies that can form appropriate or pleasing accom-
paniments to a given melody (Jansen & Povel, 2004b).
The limitation on the number of harmonic solutions
further supports the view that an unaccompanied melody
can be associated with some kind of implied harmony.
4. An implied harmony explanation of perceived
melodic similarity
Implied harmony, therefore, should provide a further
constraint upon judgments of melodic similarity.
Consider the final two notes (b,c) of a melody in C
major that imply dominant to tonic harmony. Altera-
tions to such a melody within the implied harmonic
constraint should be more similar than other changes.
Any of the chord tones of the dominant G could be used
to substitute for the b(that is, dor g), and any chord
tones in C could be used to substitute for the c(i.e. eor g)
to produce a similar melodic effect (compared with non-
chord tone replacements, such as the within-key and out-
of-key replacements discussed above). If any of the
melodic notes are chosen outside the implied harmony,
the result should be a more dissimilar melody. This
hypothesis is foreshadowed in results reported by Stoll
and Parncutt (1987), Sloboda and Parker (1985),
Holleran et al. (1995) and Trainor and Trehub (1994b).
These results also suggest that musically experienced
listeners use harmonic structure in melody cognition
whereas inexperienced listeners may not.
It appears that implied harmony plays some role in
determining similarity in Western tonal music, and that it
is a more important factor for musically skilled listeners.
What we do not know is how implied harmony and
contour interact in judgments of similarity.
Implied harmony is a complex concept because it does
not provide unique solutions for harmonic expectations
(unlike melodic displacement) – several different harmo-
nies can sometimes be used to make the same melody
sound good or pleasing. As Benward and White (1997)
remark, while there is ‘‘some leeway in the selection of
chords, a certain standard of musical communica-
tion . . . prevents you from exercising complete freedom’’
(p. 194).
5. Pitch displacement – harmonic distance
paradox
Music theoretic assumptions and empirical literature
about implied harmony can lead to predictions of
similarity judgments that conflict with predictions
derived from the effects of melodic displacement.
According to the melodic displacement perspective, the
smallest shift of a semitone in a melody with respect to
the original would produce the smallest change in
contour distance. Hence there should be little change in
similarity. However, this same shift can produce a large
(chromatic or out-of-key) departure from the implied
harmony of the original melody (see Krumhansl, 1990).
From this perspective, the clash against the implied
harmony of the original should reduce the similarity
rating. A shift by another semitone might produce a
pitch which is diatonically (within-key) different from the
implied harmony of the original. In this case the
transformed stimulus is harmonically closer than for
the first semitone change to the harmony implied by the
original, even though the melodic distance from the
164 Emery Schubert and Catherine Stevens
original has increased. The theoretical similarity response
would then appear as shown in Figure 2. In the figure,
the ‘‘3’’ refers to a distant harmony compared to the
original, the ‘‘2’’ a diatonic shift (close key) and ‘‘1’’
refers to a different note from the same implied harmony
as the original. The profile is in conflict with the
theoretical contour profile (Figure 1). The effect may be
that the two influences are additive and cancel each
other. Therefore, it is appropriate to examine each
variable level against all other variable levels (i.e. a full
factorial design).
The present study, investigates the pitch displacement –
harmonic distance paradox. It differs in several ways
from previous research that has examined the effect of
implied harmony upon melodies. First, both contour and
harmonic distance are manipulated. Second, we use
familiar melodies, in case this provides less experienced
musicians with an increased capacity to use implied
harmony information. This approach also makes the
comparative task simpler because the listener has the
original tune in memory a priori. Third, we manipulate
groups of notes rather than single target notes. Fourth,
we ask listeners to make comparisons with their long-
term memory of the melody, rather than making a direct
comparison or after a tune-learning phase. We believe
that this might help to mitigate possible confounds due
to short-term memory strategies, and therefore provide
access to tonal schemata that are used in more typical
listening situations (where short-term memory strategies
are not a central part of the cognitive organization of the
music). Finally, we investigate three levels of harmony
(most studies only examine two – within chord and out-
of chord manipulations), although within-harmony
changes are analysed separately. Since same-harmony
changes will often require relatively large interval shifts
(generally greater than or equal to a minor third), it is
possible that a confounding variable of disrupted
melodic expectation will also be manipulated (for
example, an expectation of close proximity in melody is
changed to a lack of proximity – for more information
see Narmour, 1990; Schellenberg, 1997; Huron, 2001;
Margulis, 2003). That is, a shift in pitch that maintains
the harmony of the original will usually require a
minimum of a chordal skip (an interval of a 3rd or a
4th). So there may be cases when it is not possible to have
a small change in contour with a ‘‘no-change-in-
harmony’’ condition, apart from the original melody.
To minimize this constraint, the same harmony condition
is not considered in the main factorial design, but will be
reported separately.
The hypotheses were that (1) deviations in implied
harmony and in pitch displacement contour reduce
similarity and (2) inexperienced musicians are relatively
less sensitive to changes in implied harmony than
Fig. 2. Theoretical similarity perception due to harmony (assuming implied harmony dominates response). When the consecutive pitch
distance increases by only one semitone (small pitch displacement), in some cases the harmonic distance with respect to the original
suddenly becomes large, and hence reduces similarity. It is therefore possible to increase the pitch displacement so as to, at the same
time, decrease the harmonic distance and hence increase similarity, in contrast to Figure 1.
Implied harmony, contour and expertise 165
experienced musicians. Pitch displacement and harmonic
distance were both within-subject variables and musical
expertise a between-subjects variable. The dependent
variable was perceived similarity.
6. Method
6.1 Participants
Seventy-two students were recruited from the School of
Psychology at the University of Western Sydney and the
School of Music and Music Education at the University
of New South Wales. Based on a survey, two groups were
formed: Those who reported having lessons on an
instrument greater than 9 years were placed in the
‘‘expert’’ group. Those with less than 1 year formal
training in music were placed in the ‘‘novice’’ group.
There were 44 (mean age 21.1 years, SD ¼4.7, youngest
17, oldest 42, 12 males, 32 females) and 28 (mean age
22.4 years, SD ¼8.3, youngest 18, oldest 55, 8 males, 20
females) participants in the expert and novice groups,
respectively.
6.2 Stimuli
Nursery rhyme tunes were selected which were thought
to be highly familiar to most participants: London
Bridge is Falling Down, Mary had a Little Lamb, This
Old Man and Lightly Row.
In Stage 1 (see Section 6.4), the four untransformed
melodies were used. For Stage 2 of the experiment, the
melodies were transformed by harmony and pitch
displacement. In each case, the opening bar (measure)
remained the same as the original to provide a cue for
activating the memory of the tune and to provide a
harmonic frame of reference for the experimentally
manipulated notes. Other sections were altered unless
the opening bar was repeated (as might occur in a second
musical phrase).
The Appendix shows each of the original melodies and
their transformations. The chord labels above each
system provide examples of implied harmony solutions
for the same-harmony conditions and against which the
close and distant harmony conditions would be pitted
where possible. As discussed in Sections 3.2 and 4,
several harmonizations are possible for the original
melodies. For the present study, the ‘‘template’’ harmo-
nization was selected from the three primary triads
(tonic, dominant and subdominant), providing a simple
approach for harmonizing simple melodies. This solution
was checked and agreed to by an experienced Lecturer in
harmony. Stimuli were organized such that variations in
melodic displacement from the original stimulus were
more-or-less independent of variations in harmony.
Initially, implied harmony had three values: same, close
and distant. For example, consider Lightly Row, the first
melody shown in the Appendix. The original melody
notes were substituted by chord tones from the original
implied harmony in certain sections of the test melodies
(bar 3 to 4 and bar 7 on lines 3 to 5 in the case of Lightly
Row). This transformation required some relatively large
leaps for the same harmony transformation (line 3).
Most changes required an alteration of an interval of a
third or fourth (3 to 5 semitones), setting the lower limit
of possible corresponding pitch displacement variations
(that is, they can often be displaced by no less than 3
semitones). In the close-harmony condition the notes
could be selected so as to be non-implied harmony notes,
but with a diatonic shift (that is, another note from the
same scale – ‘‘within-key’’, but not from the same
harmony). The distant-harmony condition could be, and
on some occasions was, achieved by a semitone shift, so
as to select notes that are not only non-chord tones, but
also harmonically weakly related to the original (‘‘out-of-
key’’), more weakly than the diatonic shift of the close-
harmony condition. While same-harmony alterations
were restricted to chord tones, only relatively distal
transformations in pitch were possible on most occa-
sions. However, the other two levels of harmony
manipulations could be more varied. Therefore, to
maintain a factorial design same-harmony manipulations
were analysed separately.
With this variation, two levels of pitch displacement
from the original were produced for close and distant
harmony conditions: proximal (where pitch transitions
were kept close to the original) and distal (where pitches
in the transformation were shifted further with respect to
the original). To ensure a large change in pitch contour
for the distal conditions, the direction of the melody was
changed where possible. Notice, for example, in the bot-
tom two lines of Lightly Row (shown in the Appendix)
that in the treated bars (3 to 4 and 7) the melody
descends, in opposition to the original melody direction.
The distances of the transformations were checked by
calculating the sum of the deviations between each note
of the original and the corresponding note of the
transformed melody (in units of semitones). This value
was standardized for each transformation by dividing by
the number of notes in the piece enabling comparison of
pitch displacement across stimuli. The values are shown
at the end of each line of the Appendix.
Tempi and key (starting pitch) were not randomized in
the screening stage. In Stage 2, two random variables
were introduced to reduce the chance of participants
relating the test stimuli in Stage 2 to the tempo and key
of the screening (original) stimuli used in Stage 1 (see
Section 6.4). Small variations in tempo (+10 beats per
minute) about the original, and a random key (starting
note) between þ5 and 76 semitones of the original were
chosen for each stimulus to minimize artefactual
comparison of similarity with tempo and absolute key
with Stage 1 presentations. For each example, a
166 Emery Schubert and Catherine Stevens
‘‘marimba’’ timbre was used and the nominal tempo was
set to 100 quarter beats per minute for Lightly Row,
London Bridge and This Old Man, and 120 quarter beats
for Mary Had a Little Lamb. The nominal key was set to
C major.
Therefore, for each melody, there were six presenta-
tions investigated in Stage 2 (the original melody
compared with itself, the same-harmony change, and
the 262 harmony by contour manipulations). Up to 4
melodies were selected from Stage 1, and therefore up to
24 stimuli were rated in Stage 2. Additional stimuli
were rated but are not reported here.
6.3 Equipment
The stimuli were presented to individual participants
using software written by the first author and presented
on Power Macintosh 8500 computers. The melodies were
played using a sampled marimba sound. All auditory
stimuli were played over stereo headphones at a
comfortable listening level.
6.4 Procedure
Participants sat at a computer that recorded background
information before commencing the experiment. The
experiment was conducted in two stages: screening and
testing. The first (‘‘screening’’) stage was used to ensure
that participants were familiar with each of the test
melodies. If they were unfamiliar with a melody, it was
deleted from the test stage. If they indicated familiarity
with the excerpt (‘‘Very Familiar’’ or ‘‘Familiar’’) they
were asked to select the name of the piece. This label
would be used in the second stage of the study. If the
participant selected a ‘‘not sure’’ option, the tune was
eliminated from the test stage.
In Stage 2 (testing) the participant was asked to think
about one of the melodies selected in Stage 1. The tune to
think about was prompted by the computer: ‘‘Imagine
the ‘‘original’’ tune [name of tune as indicated by
participant in Stage 1]. How similar is the tune to the
original?’’. The latter sentence pointed to a ‘‘Play’’
button which the participant clicked using the mouse to
commence hearing the test stimulus melody. The ‘‘piece
to think about’’ – ‘‘stimulus to be played’’ combination
was selected in random order without replacement.
When the participant was ready, they pressed a play
button. The participant would then rate the similarity of
the test stimulus with the one they were asked to imagine.
The response scale indicated similarity as a percentage
(visually represented scale) from 0 to 100% (with 100%
labelled ‘‘identical’’). They were encouraged to give their
first impressions, and were allowed to relisten up to four
times. The experiment lasted 25 to 30 min.
7. Results
Similarity scores were averaged across each piece for each
factor level. A repeated measures ANOVA was conducted
using the 262 within-subject factors, harmony and pitch-
displacement, and one between-subject factor, skill (novice
versus expert). There was a significant main effect for pitch-
displacement (F[1,70] ¼148.03 p50.001), but not for
harmony (F[1,70] ¼2.121 p¼0.15), as shown in Table 1.
There were significant interactions for skill with harmony
(F[1,70] ¼12.172 p¼0.001), and skill with pitch-displace-
ment (F[1,70] ¼15.568 p50.001). These two interactions
can be interpreted by examining the four plots to the right
of each pane of Figure 3. Novices rated proximal pitch
shifts as more similar to the original than distal trans-
formations (percentage similarity rating M¼60.2%
SD ¼2.37% and M¼41.39%, SD ¼2.33% respectively),
regardless of the implied harmony relationship. Expert
musicians also rated proximal transformations as being
more similar than distal transformations, but to a lesser
extent (M¼53.14% SD ¼1.89% and M¼43.51%,
SD ¼1.86% respectively). However, expert musicians
made further distinctions in similarity which corresponded
to the implied harmony manipulations. Each level of the
262 factors (harmony, pitch displacement) produced a
significant difference which, from most similar to least,
Table 1. Pairwise comparisons of similarity response for harmony and contour by skill.
Skill level Mean difference Std. Sig.(a)
95% Confidence interval for
difference
b
HARMONY Lower bound Upper bound
Novice Close Distant 71.922 1.478 0.198 74.87 1.026
Expert Close Distant 4.676
a
1.179 0 2.324 7.027
CONTOUR Lower bound Upper bound
Novice Proximal Distal 18.858
a
1.83 0 15.208 22.508
Expert Proximal Distal 9.622
a
1.46 0 6.711 12.534
a
Significant at p¼0.05.
b
Adjusted for multiple comparisons: least significant difference.
Implied harmony, contour and expertise 167
were close-proximal, distant-proximal, close-distal, dis-
tant-distal, as shown in Figure 3.
Figure 3 also shows the mean response to ratings of
the original melody by level of expertise. The results
are collapsed across the random tempo and starting
note (key) levels. The results demonstrate higher
similarity of ratings of original melodies by the musical
expert group (M¼95.2%, SE ¼0.665%) compared
with the novice group (M¼85.85%, SE ¼1.956%).
The experts seemed to reach a ceiling, hence the small
standard error.
Finally, responses to within harmony changes were
examined. As shown in Figure 3, changing only the
harmony note (that is, making pitch more distal with
respect to original without changing harmony) produced
a drop in similarity scores for both experts (M¼51.61%,
SE ¼1.23%) and novices (M¼54.23%, SE ¼1.97%).
Although similarity scores obtained from novices are
slightly higher than experts for same-harmony manip-
ulation, the pitch shift factor alone demonstrates a trend
for the novices which can be wholly determined by
contour, which is, in decreasing order of similarity:
Fig. 3. Perceived similarity by implied harmony and contour for experts and novices. The yaxis indicates average perceived similarity
percentage. Novice n¼28; Expert n¼44. Error bar is +1 SE.
168 Emery Schubert and Catherine Stevens
proximal contour shift, within harmony change (which
constitutes a contour shift in between proximal and
distal) and distal contour shift. For the expert group, a
rough equivalence or trade-off can be seen between the
same harmony condition and proximal contour-distant
harmony condition. These findings reinforce the notion
that novices almost ignore the influence of implied
harmony upon similarity judgements, whereas for music
experts the two factors have some interplay, with
contour, nevertheless, being the dominating factor.
8. Discussion and conclusion
Consistent with previous literature, the results suggest
that melodic contour plays a major role in the organiza-
tion of melodies in long-term memory, and more so than
deeper level implied harmony, supporting Lamont and
Dibben’s (2001) finding that contour (surface structure)
seems to be more important in determining similarity
than deeper structure (in this case, implied harmonic
relationships). In addition, the experiment demonstrates
utility in investigating contour as a displacement with
respect to its difference (in semitones) from a memorized
melody. The greater the displacement of a melody from
the source melody, the greater the difference in perceived
similarity. The experiment indicates that implied har-
mony plays a secondary role, if any, in determining
similarity, and that the information is used by more
experienced musicians in agreement with Holleran et al.
(1995). The results of the Holleran et al. study can be
further refined based on the data of the present study.
First, for musically experienced individuals, diatonic
(close harmony) and chromatic (distant harmony)
changes produced some perceptual alterations of the
melody. Consistent with music theory, distant harmony
shifts are considered more different than are close
harmony shifts with respect to the original melody. This
finding is also consistent with that of Massaro et al. (1980)
and Trainor and Trehub (1994b). Second, Holleran et al.
reported that less experienced musicians are still influ-
enced by implied harmony but to a lesser extent than
highly skilled musicians. In their study, the low-skilled
listeners had at least 5 years of formal musical training. In
our experiment, the novices had one year or less formal
musical training. This suggests that there may be some
kind of sliding scale in the use of implied harmony
information, where novices use little of this information
in determining the similarity of two melodies.
The present study also extends previous findings to
melodies stored in long-term memory, and not just
specially composed melodies directly compared with their
modifications (Holleran et al. 1995; Trainor & Trehub,
1994b), or less familiar melodies (Sloboda & Parker,
1985). Further, it has examined the factors of contour and
harmony manipulations and their interaction.
The overlapping similarity ratings for the distant and
close harmony conditions could be taken to imply that
novice listeners are unable to make fine distinctions
between within-key (diatonic) and out-of-key (chromatic)
changes. The data suggest that for novice listeners, all
dissonances are equal. However, the within-chord (same-
harmony) change is rated slightly lower in similarity than
the proximal pitch shift condition, meaning that even
when the alteration produces greater consonance, the
response still has a lower similarity rating. Two explana-
tions of this result are proposed. (1) Harmonic distance
for novices is a red herring, and imposes no systematic
alteration of similarity judgments (‘‘dissonance is dis-
sonance’’), or (2) there is a third variable, which could not
be controlled in the ‘‘within-chord’’ change of the current
study – namely voice leading – which might cause an
artificial lowering of similarity scores. Indeed, the at-
times unusual leaps introduced in the same-harmony
condition could have been responsible for the drop in
similarity, masking the effect of harmonic distance.
Further research is required to determine which explana-
tion holds. It may also be worth investigating whether
these effects are peculiar to the simple pieces chosen, or
whether they generalize to more complex kinds of music
(such as those using chromaticism and modulation) where
the question of harmonization itself becomes still more
complex. The question of whether implied harmony
would assume a more or less important role in judgements
of similarity in other contexts becomes quite interesting.
The lower similarity rating of the original melody
compared with the memory of the original melody for
the novices is an anomalous finding which is consistent
with Holleran et al. (1995). Since tempo and key were
varied in the present experiment, it would make sense to
predict that listeners with good pitch and tempo memory
(represented by the expert group) might provide lower
similarity ratings for the original stimuli with altered
tempi and keys. The reverse was the case. It might be that
novices are less confident in identifying precise matching
because they are less able to maintain as precise a
representation of the melody under investigation. It
might also be that expert listeners are more willing to
assimilate features of a melody which are well conserved
under transformation, such as starting note (key) and
tempo (Levitin, 1999). In fact, it is also conceivable that
novice listeners may have quite good pitch memory
(Schellenberg & Trehub, 2003), and may be more
sensitized to these effects. This issue is a point for further
investigation.
The results present a dilemma in that the prediction
derived from music theory seems to apply to experienced
musicians, rather than the general, enculturated popula-
tion. For example, since adults have more experience, they
should be more likely to use implied harmony to help
organize melodic material than would children. Trainor &
Trehub (1994a) reported that by the age of 7 years children
Implied harmony, contour and expertise 169
were responding at about the same level as adult
(mostly novice) listeners in their use of implied harmony
information (see also Costa-Giomi, 2003). The question
then remains: Do musicians develop a strategy for
detecting dissimilarity? If so, the present study may be
detecting nothing more than a pre-conditioned response,
or a more complete enculturation of the rules of harmony.
In post-experiment surveys there was no evidence to
support the notion that musically experienced participants
were using any conscious strategy that related to harmonic
relationships. It seems that the difference between musician
and non-musician is a consequence of training and a more
thoroughly absorbed internalization of harmonic relation-
ships and expectancies. It suggests that deeper level
structures in music may be defined in terms of encultura-
tion. Ultimately, the implication of this finding for melodic
system retrieval and for cognitive architecture is that there
are important individual differences, particularly among
novice and expert musicians, in the way musical informa-
tion is retained and retrieved.
The present findings also have application to the
development of automated retrieval systems in that they
suggest the need for an additional level of complexity in
designing such systems, one which accounts for the
underlying harmony of the melodic information. While a
semitone shift in some parts of a familiar melody may
not violate the contour of the melody, it will clash with
the expected harmonic flow (Jansen & Povel, 2004b;
Povel & Jansen, 2002), and may be considered less
similar than its seed if an experienced listener is being
modelled or simulated. Modifications to models such as
those discussed by Kim et al. (2000) and Holleran et al.
(1995, p. 739) may require a ‘‘musical experience’’ input
level. If future research supports the relative importance
of the role of harmonic structure with respect to contour
in the organization of melodic material, algorithms such
as those proposed by Krumhansl (1990) for measuring
key-distance, Lerdahl (1996, 2001) for measuring tonal
tension, and Jansen and Povel’s (2004b) on-line harmo-
nic processing model may prove useful in providing
information retrieval systems with resources for identify-
ing harmonic relationships implied in melodies.
The reason for the greater reliance on harmony
information by experienced listeners may be due to the
development of more analytic listening and feature
extraction and feature weighting processes. However, it
may also be an effect of efficient memory strategies. If
experienced listeners have much musical material to
store, process and retrieve, they require more refined
strategies, such as chunking, for organizing this large
amount of information (e.g. Ericsson, 2003; Ericsson &
Delaney, 1999; McAuley et al. 2004). The simplest,
directional contour coding strategy would be of little use
if the number of melodies requiring storage and
processing becomes very large (see Kim et al. 2000).
Some ‘‘psychologically different’’ melodies will even-
tually be found which may have identical ‘‘direction’’
contours and, as the number increases even further,
displacement information may also be insufficient. Thus
alternative strategies may emerge, such as attending to
harmonic implication. Experts are likely to have flex-
ibility in global or analytic strategies depending on task,
instructions and context. Novices are more limited in
available strategies and cognitive resources. Future work
is required to determine whether novices are insensitive
to implied harmony, or whether they simply do not use
their implicit knowledge of it. Infant sensitivity to out-of-
harmony changes (Trainor & Trehub, 1992) suggests that
infants and adults do encode harmonic information. By
building on this finding, the present study suggests that
novice listeners may encode this information but cannot
or do not use it to the same extent as highly skilled
musicians.
Acknowledgment
This research was supported by a University of Western
Sydney Research Encouragement Grant awarded to the
second author. We are grateful to Colin Watts from the
School of Music and Music Education at the University
of New South Wales for his assistance in the preparation
of the musical examples and to Melinda Gallagher for
assistance with data collection.
References
Anderson, J.R. & Schooler, L.J. (2000). The adaptive nature
of memory. In E. Tulving and F.I.M. Craik (Eds.)
Handbook of Memory, pp. 557 – 570. New York: Oxford
University Press.
Bartlett, J.C. & Dowling, W.J. (1988). Scale structure and
similarity of melodies. Music Perception,5, 285 – 314.
Benward, B. and White, G. (1997). Music in Theory and Prac-
tice,Vol.1,6thEdn.Madison,WI:BrownandBenchmark.
Bharucha, J.J. & Krumhansl, C.L. (1983). The represen-
tation of harmonic structure in music: hierarchies
of stability as a function of context. Cognition,13,63–
102.
Bigand, E. (1993). Abstraction of two forms of underlying
structure in a tonal melody. Psychology of Music,18,
45 – 59.
Byrd, D. & Crawford, C. (2002). Problems of music infor-
mation retrieval in the real world. Information Processing
& Management,38, 249 – 272.
Cambouropoulos, E. (2000). Melodic cue abstraction,
similarity and category formation: a computational
approach. Retrieved 1 August 2003 from http://citeseer.
nj.nec.com/cambouropoulos00melodic.html
Cambouropoulos, E. & Widmer, G. (2000). Melodic
clustering: motivic analysis of Schumann’s Trumerei.
Paper presented at Proceedings of the III Journes d’
Informatique Musicale, Bordeaux, France. Retreived 15
July 2003 from http://citeseer.nj.nec.com/423201.html
170 Emery Schubert and Catherine Stevens
Carterette, E.C., Kohl, D.V., & Pitt, M.A. (1986). Simila-
rities among transformed melodies: the abstraction of
invariants. Music Perception,3, 393 – 410.
Costa-Giomi, E. (2003). Young children’s harmonic percep-
tion. Neurosciences & Music,999, 477 – 484.
Cross, I., Howell, P., & West, R. (1983). Preferences for
scale structure in melodic sequences. Journal of Experi-
mental Psychology: Human Perception & Performance,
9(3), 444 – 460.
Davidson, L. (1985). Tonal structures of children’s early
songs. Music Perception,2, 361 – 374.
Deliege, I. (2001). Introduction: similarity perception 5–4
categorization 5–4cue abstraction. Music Perception,
18, 233 – 243.
Dowling, W. (1978). Scale and contour: two components of
a theory of memory for melodies. Psychological Review,
85(4), 341 – 354.
Dowling, W.J. (1988). Tonal structure and children’s early
learning of music, In J.A. Sloboda (Ed.), Generative
Processes in Music, pp. 113 – 128. Oxford: Clarendon
Press.
Dowling, W.J. (1991). Tonal strength and melody recogni-
tion after long and short delays. Perception and Psycho-
physics,50, 305 – 313.
Dowling, W. (1994). Melodic contour in hearing and
remembering melodies. In R. Aiello & J. A. Sloboda
(Eds.), Musical Perceptions. pp. 173 – 190. London:
Oxford University Press.
Dowling W.J. & Bartlett, J.C. (1981). The importance of
interval information in long-term memory for melodies.
Psychomusicology,1, 30 – 49.
Edworthy, J. (1985). Interval and contour in melody
processing. Music Perception,2, 375 – 388.
Ericsson, K.A. (2003). Exceptional memorizers: made, not
born. Trends in Cognitive Sciences,7, 233 – 235.
Ericsson, K.A. & Delaney, P.F. (1999). Long-term
working memory as an alternative to capacity models of
working memory in everyday skilled performance. In
A. Miyake and P. Shah (Eds.), Models of Working
Memory: Mechanisms of Active Maintenance and Execu-
tive Control, pp. 257 – 297. Cambridge: Cambridge
University Press.
Halpern, A.R. (1984a). Organization in memory for familiar
songs. Journal of Experimental Psychology: Learning,
Memory, and Cognition,10, 496 – 512.
Halpern, A.R. (1984b). Perception of structure in novel
music. Memory and Cognition,12, 163 – 170.
Halpern, A.R. (1989). Memory for the absolute pitch
of familiar songs. Memory and Cognition,17, 572 –
581.
Halpern A.R., Bartlett J.C. & Dowling W.J. (1998).
Perception of mode, rhythm, and contour in unfamiliar
melodies: effects of age and experience. Music Perception,
15, 335 – 355.
He
´bert, S. & Peretz, I. (1997). Recognition of music in
long-term memory: are melodic and temporal
patterns equal partners? Memory & Cognition,25, 518 –
533.
Hofmann-Engl, L. (2001). Towards a cognitive model of
melodic similarity. In J.S. Downie and D. Bainbridge
(Eds.), Proceedings of the Second Annual International
Symposium on Music Information Retrieval, pp. 143 –
151. Retrieved 15 June 2003 from http://ismir2001.
ismir.net/pdf/hofmann-engl.pdf
Holleran, S., Jones, M.R. & Butler, D. (1995). Perceiving
implied harmony: the influence of melodic and harmonic
context. Journal of Experimental Psychology-Learning
Memory & Cognition,21(3), 737 – 753.
Huron, D. (2001). Tone and voice: a derivation of the rules
of voice-leading from perceptual principles. Music
Perception,19, 1 – 64.
Jansen, E. & Povel, D.J. (2004a). Perception of arpeggiated
chord progressions. Musicae Scientiae,8(1), 7 – 52.
Jansen, E. & Povel, D.J. (2004b). The processing of chords
in tonal melodic sequences. Journal of New Music
Research,33(1), 31 – 48.
Jones, M.R., Summerell, L., & Marshburn, E. (1987).
Recognizing melodies: a dynamic interpretation. Quar-
terly Journal of Experimental Psychology,39A, 89 – 121.
Kim, Y.E., Chai, W., Garcia, R. & Vercoe, B. (2000).
Analysis of a contour-based representation for melody.
Retrieved 15 June 2003 from www.media.mit.edu/
chaiwei/papers/Kim.pdf
Krumhansl, C. (1990). Cognitive Foundations of Musical
Pitch. Oxford: Oxford University Press.
Krumhansl, C.L. & Keil, F.C. (1982). Acquisition of the
hierarchy of tonal functions in music. Memory &
Cognition,10(3), 243–251.
Lamont, A. & Dibben, N. (2001). Motivic structure and the
perception of similarity. Music Perception,18, 245 – 274.
Lerdahl, F. (1996). Calculating tonal tension. Music
Perception,13, 319 – 363.
Lerdahl, F. (2001). Tonal Pitch Space. New York: Oxford
University Press.
Levitin, D.J. (1999). Memory for musical attributes. In
P.R. Cook (Ed.), Music, Cognition and Computerized
Sound: An Introduction to Psychoacoustics, pp. 209 – 227.
Cambridge, MA: MIT Press.
Margulis, E.H. (2003). Melodic expectation: a discussion
and model. Dissertation Abstracts International,64(4-A),
1127.
Massaro, D.W., Kallman, H.J. & Kelly, J.L. (1980). The
role of tone height, melodic contour, and tone chroma in
melody recognition. Journal of Experimental Psychology:
Human Learning and Memory,6, 77 – 90.
McAdams, S. & Matzkin, D. (2001). Similarity, invariance,
and musical variation. Biological Foundations of Music
Annals of the New York Academy of Sciences,930, 62 – 76.
McAuley, J.D., Stevens, C., & Humphreys, M.S. (2004).
Play it again: did this melody occur frequently or was it
heard more recently? The role of stimulus familiarity in
the formation of episodic memories for music. Acta
Psychologica,116, 93 – 108.
Narmour, E. (1990). The Analysis and Cognition of Basic
Melodic Structures: The Implication-Realization Model.
Chicago: University of Chicago Press.
Implied harmony, contour and expertise 171
Povel, D.J. & Jansen, E. (2002). Harmonic factors in the
perception of tonal melodies. Music Perception,20(1),
51 – 85.
Schellenberg, E.G. (1997). Simplifying the implication-
realization model of melodic expectancy. Music Percep-
tion,14, 295 – 318.
Schellenberg, E.G., & Trehub, S.E. (2003). Good pitch
memory is widespread. Psychological Science,14(3), 262 –
266.
Schoenberg, A. (1967). In G. Strang (Ed.), with the
collaboration of L. Stein (Ed.). Fundamentals of Musical
Composition. London: Faber.
Sloboda, J.A. & Parker, D.H.H. (1985). Immediate recall of
melodies. In P. Howell, I. Cross & R. West (Eds.), Musical
Structure and Cognition. London: Academic Press.
Stoll, G. & Parncutt, R. (1987). Harmonic relationship in
similarity judgments of nonsimultaneous complex tones.
Acustica,63(2), 111 – 119.
Tan, N., Aiello, R., & Bever, T. (1981). Harmonic structure
as a determinant of melodic organization. Memory and
Cognition,9, 535 – 539.
Tillmann, B., Bharucha, J.J., & Bigand, E. (2000). Implicit
learning of tonality: a self-organizing approach. Psycho-
logical Review,107(4), 885 – 913.
Tillmann, B. & Bigand, E. (2004). Further investigation of
harmonic priming in long contexts using musical timbre
as surface marker to control for temporal effects.
Perceptual & Motor Skills,98(2), 450 – 458.
Trainor, L.J., Desjardins, R. N., & Rockel, C. (1999). A
comparison of contour and interval processing in
musicians and nonmusicians using event-related poten-
tials. Australian Journal of Psychology,51(3), 147 – 153.
Trainor, L.J., McDonald, K.L., & Alain, C. (2002). Auto-
matic and controlled processing of melodic contour and
interval information measured by electrical brain activity.
Journal of Cognitive Neuroscience,14(3), 430 – 442.
Trainor, L.J. & Trehub, S.E. (1994a). A comparison of
infants’ and adults’ sensitivity to Western musical
structure. Journal of Experimental Psychology: Human
Perception and Performance,18, 394 – 402.
Trainor, L.J. & Trehub, S.E. (1994b). Key membership and
implied harmony in Western tonal music: developmental
perspectives. Perception and Psychophysics,56, 125 – 132.
Trehub, S.E. (1990). Human infants’ perception of auditory
patterns. International Journal of Comparative Psychol-
ogy,4, 91 – 110.
Trehub, S.E. (2001). Musical predispositions in infancy. In
R.J. Zatorre & I. Peretz (Eds.), The Biological Founda-
tions of Music, Annals of the New York Academy of
Sciences, 930, 439 – 442.
Trehub, S.E., Bull, D., & Thorpe, L.A. (1984). Infants’
perception of melodies: the role of melodic contour.
Child Development,55, 821 – 830.
Van Egmond, R., & Povel, D.J.L. (1996). Perceived
similarity of exact and inexact transpositions. Acta
Psychologica,92, 283 – 296.
Welker, R.L. (1982). Abstraction of themes from melodic
variations. Journal of Experimental Psychology: Human
Perception and Performance,8, 435 – 447.
Zhu, Y. & Kankanhalli, M. (2002). Similarity matching of
continuous contours for humming querying of melody
databases. LIT Technical Report, February. Retrieved 15
June 2005 from www.comp.nus.edu.sg/cs4241/
TR2002.pdf
172 Emery Schubert and Catherine Stevens
Appendix: Stimuli used in the experiment
Numbers to the right of each line indicate the average
deviation of the pitches with respect to the original (in
units of semitones, or chroma). In the screening stage,
only the original stimulus, shown as the top line, was
used. In the test stage all stimuli were compared against a
participant’s memory of the tune represented by the
original. An implied harmony solution is shown as chord
labels above the original melody (see text). The braces
below each system indicate the sections of bars in which
test stimuli notes were transformed.
Implied harmony, contour and expertise 173
174 Emery Schubert and Catherine Stevens
