Content uploaded by Heather C Lench
Author content
All content in this area was uploaded by Heather C Lench on Jun 25, 2014
Content may be subject to copyright.
Content uploaded by Heather C Lench
Author content
All content in this area was uploaded by Heather C Lench on Feb 17, 2014
Content may be subject to copyright.
Discrete Emotions Predict Changes in Cognition, Judgment,
Experience, Behavior, and Physiology: A Meta-Analysis of Experimental
Emotion Elicitations
Heather C. Lench, Sarah A. Flores, and Shane W. Bench
Texas A&M University
Our purpose in the present meta-analysis was to examine the extent to which discrete emotions elicit
changes in cognition, judgment, experience, behavior, and physiology; whether these changes are
correlated as would be expected if emotions organize responses across these systems; and which factors
moderate the magnitude of these effects. Studies (687; 4,946 effects, 49,473 participants) were included
that elicited the discrete emotions of happiness, sadness, anger, and anxiety as independent variables with
adults. Consistent with discrete emotion theory, there were (a) moderate differences among discrete
emotions; (b) differences among discrete negative emotions; and (c) correlated changes in behavior,
experience, and physiology (cognition and judgment were mostly not correlated with other changes).
Valence, valence–arousal, and approach–avoidance models of emotion were not as clearly supported.
There was evidence that these factors are likely important components of emotion but that they could not
fully account for the pattern of results. Most emotion elicitations were effective, although the efficacy
varied with the emotions being compared. Picture presentations were overall the most effective elicitor
of discrete emotions. Stronger effects of emotion elicitations were associated with happiness versus
negative emotions, self-reported experience, a greater proportion of women (for elicitations of happiness
and sadness), omission of a cover story, and participants alone versus in groups. Conclusions are limited
by the inclusion of only some discrete emotions, exclusion of studies that did not elicit discrete emotions,
few available effect sizes for some contrasts and moderators, and the methodological rigor of included
studies.
Keywords: emotion elicitation, emotion manipulation, emotion theory, discrete emotion
Supplemental materials: http://dx.doi.org/10.1037/a0024244.supp
Although emotion has long been recognized as a driving force in
human psychology, it has only recently become a topic of empir-
ical psychological investigation. Despite their recent appearance
on the empirical scene, emotions have taken center stage in many
investigations as independent variables that influence outcomes
ranging from reaction time to prosocial behavior. Emotional con-
structs are central to modern theoretical models in a number of
social sciences, including psychology (Bower & Forgas, 2000;
Brown, 2000; Epstein, 1994), economics (Kahneman, 2003; Le-
rner & Keltner, 2000; Loewenstein, 1996), and philosophy (Haidt,
2001; Rozin, 1999). There has been substantial advancement in the
past three decades in the study of the consequences of emotion, but
empirical investigation of emotion has been delayed by disagree-
ments over the most effective theoretical perspective for capturing
differences among emotional states and methodological uncer-
tainty regarding how to elicit emotions effectively. In an attempt to
resolve some of these uncertainties, the purpose of this review was
to examine: (a) whether the discrete emotions of happiness, sad-
ness, anger, and anxiety elicit changes in cognition, judgment,
experience, behavior, and physiology; (b) whether these changes
are correlated as would be expected if emotions organize responses
across these systems; and (c) which factors moderate the magni-
tude of these effects.
Major Theories of Emotion
Our primary goal in this review was to assess whether discrete
emotions elicit changes in cognition, judgment, experience, behav-
ior, and physiology. The expectation that discrete emotions would
have unique effects on multiple outcomes arises from a function-
alist perspective that describes emotion as an evolutionarily adap-
tive response that organizes cognitive, experiential, behavioral,
and physiological reactions to changes in the environment (e.g.,
Ekman, 1992; Frijda, 1987; Izard, Levinson, Ackerman, Kogos, &
Blumberg, 1998; Lerner & Keltner, 2001; Mauss, Levenson, Mc-
Carter, Wilhelm, & Gross, 2005; Pinker, 1997; Plutchik, 2000;
Rottenberg, Ray, & Gross, 2007; Tomkins & McCarter, 1964).
Most versions of this perspective include the proposition that each
This article was published Online First July 18, 2011.
Heather C. Lench, Sarah A. Flores, and Shane W. Bench, Department of
Psychology, Texas A&M University.
Thanks to the many dedicated research assistants who helped to find,
code, and enter information for this review, particularly Cian Brown, and
to Christopher Berry, JoAnn Prause, and Martin Safer for comments on
earlier drafts.
Correspondence concerning this article should be addressed to Heather
C. Lench, Department of Psychology, Texas A&M University, College
Station, TX 77843-4235. E-mail: hlench@tamu.edu
Psychological Bulletin © 2011 American Psychological Association
2011, Vol. 137, No. 5, 834– 855 0033-2909/11/$12.00 DOI: 10.1037/a0024244
834
discrete emotion elicits changes in cognition (e.g., narrowing of
attention on a tiger in the distance), judgment (e.g., the risk
perceived in the environment), experience (e.g., the recognition
that one is afraid), behavior (e.g., a tendency to run away), and
physiology (e.g., increased heart rate and respiration) that are
adapted to facilitate a response to the types of environmental
changes that elicit that emotion. Discrete emotion theories vary,
however, in whether they focus on changes in all of these systems
or different subsets of these systems. For example, the appraisal-
tendency framework stresses changes in cognition and judgment
that accompany discrete emotions (Lerner & Keltner, 2001),
whereas differential emotions theory stresses the co-occurrence of
changes in physiology, emotional expressions, and subjective ex-
perience (Izard, 1971; Izard & Abe, 2004). Despite the growing
prominence of the functionalist perspective and increasing agree-
ment that emotions elicit changes across multiple systems, there is
disagreement regarding this proposition and, in some cases, mixed
findings exist. There has been no comprehensive review of em-
pirical findings to address the degree to which there is support for
the proposition.
Evidence for the proposition that each discrete emotion elicits
changes in cognition, judgment, behavior, experience, and physi-
ology is mixed. Some studies have found that discrete emotions
lead to changes in outcomes consistent with the predicted adaptive
function of the emotion (Frijda, 1987; Lench & Levine, 2005;
Lerner & Keltner, 2001). Limited evidence suggests that these
changes may co-occur following an emotion-eliciting event, al-
though the co-occurrence of changes across systems may change
across the life span and contexts (Izard & Abe, 2004; Izard,
Hembree, & Huebner, 1987; Mauss et al., 2005; Shiller, Izard, &
Hembree, 1986). Although there is disagreement about the specific
elements within situations that elicit emotional responses, happi-
ness is generally considered to arise from a success (e.g., Fredrick-
son, 2001; Isen, 2000), sadness from failure or loss with no hope
of reinstating the goal (e.g., Frijda, 1987; Gross & Levenson, 1995;
Lerner & Keltner, 2001), anger from failure with the possibility of
goal reinstatement (i.e., the goal is still possible with additional
effort; e.g., Bodenhausen, Mussweiler, Gabriel, & Moreno, 2001;
Gross & Levenson, 1995; Harmon-Jones & Sigelman, 2001), and
anxiety from the anticipation of threats to important goals (e.g.,
Frijda, 1987; Gross & Levenson, 1995). The use of specific emo-
tion terms varies considerably among theories and perspectives,
and the definitions of discrete emotions used in this review con-
ceptually overlap with some other definitions (e.g., some perspec-
tives describe fear as a discrete emotional reaction to anticipated
threats and anxiety as a mood state; some perspectives describe
depression as the emotional reaction to loss rather than sadness).
To the extent that emotions represent discrete constructs, each
emotion should differ from other emotions (and neutral groups) on
measures of cognition, judgment, experience, behavior, and phys-
iology, and the differences should co-occur across these systems.
Alternative theories of emotion predict, and there is empirical
evidence to suggest, that valence or valence combined with arousal
captures the important differences among emotions (e.g., Barrett,
1998; Russell, 1980; Watson & Tellegen, 1985). If valence effec-
tively captured differences among emotions, sadness, anger, and
anxiety would be expected to have similar effects on outcomes
because they are emotions characterized by negative valence.
Although there are some discrepancies among models that include
both valence and arousal, most models place anger and anxiety
very close to one another in terms of valence (negative) and
arousal (high; Feldman-Barrett & Russell, 1998; Larsen & Diener,
1992; Russell, 1980; Watson & Tellegen, 1985). If valence and
arousal effectively captured differences among emotions, anger
and anxiety would be expected to have similar effects on out-
comes. These accounts sometimes further posit that observed
differences among discrete emotions arise from cultural expecta-
tions regarding emotions (e.g., people expect anger and anxiety to
be different) rather than from a true difference in emotional reac-
tions (Barrett & Russell, 1999). This possibility cannot be directly
addressed in the present investigation. However, if differences
were due to beliefs about emotions, differences among discrete
emotions would be expected to be most evident in self-reports of
emotional experience that are at least partially based on general
beliefs about emotional experience (Robinson & Clore, 2002) and
least evident for other outcomes, such as behavioral and physio-
logical reactions.
Finally, some models of emotion suggest, and studies have
found, that approach and avoidance motivations determine re-
sponses to situations, although there is debate about the emotions
associated with each motivation (Bechara, Damasio, Tranel, &
Damasio, 1997; Carver, 2004; Gray, 1972). In several models,
positive emotions reflect approach motivations and negative emo-
tions reflect avoidance motivations (Bechara et al., 1997; Gray,
1972). The same predictions would be derived from these models
as from the valence models described previously. In other models,
happiness and anger reflect approach motivations and sadness and
anxiety reflect avoidance motivations (Bodenhausen, Sheppard, &
Kramer, 1994; Harmon-Jones, 2003; Henriques & Davidson,
1991). If these distinctions effectively captured differences among
emotions, one would expect emotions of a similar motivational
direction (happiness and anger; sadness and anxiety) to have
similar effects on outcomes and emotions of different motivational
directions (happiness, anger vs. sadness, anxiety) to have different
effects on outcomes.
The present investigation allowed for a test of the general
proposition that discrete emotions lead to changes in cognitive,
judgment, experiential, behavioral, and physiological outcomes
and for an examination of support for a number of predictions
about the most efficient way to capture differences among emo-
tions. It is important to note that there are some predictions related
to the discrete emotions perspective that cannot be evaluated in
this review, including the degree to which changes in different
systems facilitate a functional response to the environment. Thus,
our primary purpose in this review was to examine the effects of
discrete emotions across outcomes and the extent to which these
changes co-occur across systems.
Major Types of Elicitations and Related Issues
The other primary goal in the present review was to provide
practical information on moderators of these effects, in particular
factors associated with emotion elicitations. There is great varia-
tion in the types of elicitations used to manipulate emotional states
in experimental research designs, and researchers must either
adopt a method from another researcher or develop their own
idiosyncratic methods. Often, they must make this choice with
little empirical guidance regarding the effectiveness of different
835
DISCRETE EMOTION META-ANALYSIS
methods (Rottenberg et al., 2007). Several reviews have been
conducted on positive versus negative mood elicitations and have
generally found that most techniques to induce mood are effective
(Gerrards-Hesse, Spies, & Hesse, 1994; Westermann, Spies, Stahl,
& Hesse, 1996), but there have been no reviews of elicitations
of discrete emotions. In the text below, general classifications of
emotion elicitation techniques are defined and a brief overview of
practices and potential issues is presented for each type of elici-
tation. It is important to note that the limitations of the methods
used to elicit emotions necessarily curtail the validity of the effects
included in the present review.
Film
Film emotion elicitations typically consist of showing partici-
pants a brief video clip, often a portion of a feature-length film,
that is intended to elicit discrete emotions. This elicitation is an
attractive alternative, because video clips are relatively short, in-
tuitively powerful, and easily accessible and the clips and the
procedure for viewing them can be standardized across partici-
pants (Kring & Gordon, 1998; Rottenberg et al., 2007). Another
advantage of film elicitations is the availability of standardized
video clips that reliably elicit different emotional reactions (e.g.,
Gross & Levenson, 1995; Philippot, 1993; Rottenberg et al., 2007).
There are also several potential disadvantages to the use of video
clips. Some emotions, such as anxiety, appear particularly difficult
to elicit with films (Gross & Levenson, 1995). Further, individual
differences and prior experiences with the films may moderate
participants’ reactions to video clips (Gross, 1998; Larsen &
Ketelaar, 1991). An additional issue regarding the use of films is
that they have typically been validated by self-reports of the
intensity of experienced emotion. It is unclear whether films elicit
changes in other systems, particularly as reactions in cognitive,
judgment, experiential, behavioral, and physiological outcomes are
often loosely coupled (e.g., Bradley & Lang, 2000; Mauss et al.,
2005; Weinstein, Averill, Opton, & Lazarus, 1968). Further, films
are cognitively complex stimuli that may prime cognitions in
addition to those related to the emotion and that depict emotional
situations happening to another person in a fictional setting. To be
effective, such elicitations may require that participants be willing
to engage in the fictional reality, and it is debatable whether
emotions elicited by film viewing are similar to daily emotional
experience (even the most devout horror buff would presumably
not enjoy being chased by an axe-wielding maniac; e.g., Silvia,
2005; Walters, 1989). An additional issue with the use of video
clips is that, although the presentation of each clip can be stan-
dardized, it is difficult to standardize across clips because films
often differ from one another in multiple details that may impact
outcomes (e.g., the number of people, colors used, background
music).
Pictures
The supraliminal presentation of pictures is also used to elicit
emotional reactions. The most commonly used pictures come from
the International Affective Pictures System (IAPS; Bradley &
Lang, 2007; Center for the Study of Emotion and Attention, 1995;
Lang, 1995), a set of pictures that has been standardized on
valence, arousal, and dominance dimensions. The advantages and
disadvantages with picture elicitations are similar to those associ-
ated with the use of video clips. An additional disadvantage to the
use of pictures from the IAPS to elicit discrete emotions is that the
images are intended to elicit changes in valence and arousal, not
discrete emotions, and individuals vary in the emotional responses
they report to the images (Bradley & Lang, 2007). There have,
however, been recent attempts to identify standardized sets of
images that elicit discrete emotions (Mikels et al., 2005).
Priming
Priming elicitations present stimuli outside of participants’ con-
scious awareness; they often consist either of subliminal presen-
tation of pictures or words or of exposure to words through subtle
tasks, such as unscrambling anagrams or crossword puzzles. The
assumption is that these stimuli activate or “prime” related con-
structs in memory and thus elicit the emotion related to the stimuli
(e.g., Bargh & Gollwitzer, 1994). The advantage to priming emo-
tions is that the elicitation occurs outside of awareness and that
demand characteristics therefore should be minimal. Potential dis-
advantages include ambiguity about whether emotions are primed
or are simply related cognitions, as well as the lack of standard-
ization between conditions and studies. Researchers frequently
construct their own priming stimuli, based on stimuli that have
been successful in other priming studies. They typically use primes
that appear intuitively linked to the emotion concept (e.g., a picture
of an extended middle finger or a crossword puzzle containing
words such as hot and frustrate to prime anger).
Music
In music elicitations, participants listen to musical selections,
sometimes following instructions that they should attempt to sim-
ulate the emotion intended by the music (Clark, 1983; Va¨stfja¨ll,
2002). Music elicitations were developed to elicit mood states and
appear to do so effectively when positive and negative mood states
are compared with one another (the effects are less consistent if
mood states are compared with a neutral group and with behavioral
measures versus self-report measures; Albersnagle, 1988; Clark,
1983; Kenealy, 1988; Martin, 1990; Va¨stfja¨ll, 2002). There is
disagreement regarding whether music can theoretically elicit dis-
crete emotions (defined as relatively brief, intense states); how-
ever, music has been used with the intention to elicit discrete
emotions, including anger and anxiety, and these studies are in-
cluded in the present review (e.g., Va¨stfja¨ll, 2002). Like films and
pictures, music elicitations are attractive because the presentation
can be standardized across participants, and there are selections
that have been developed to reliably elicit particular states. The
disadvantages to the use of music are similar to those for films and
pictures, including the difficulty of standardizing music selections
across emotional conditions and the potential need for participants
to willingly engage with the stimuli (Scherer & Zentner, 2008;
Silvia, 2005; Walters, 1989). Additional issues arise from the
practice of including instructions to simulate the emotion, because
these instructions may produce demand effects and participants
may vary in strategies they employ to achieve the emotional state
(Kenealy, 1988; Mayer, Gayle, Meehan, & Haarman, 1990; Va¨st-
fja¨ll, 2002). Another potential disadvantage to the use of music is
that most selections are classical music (likely because they have
836
LENCH, FLORES, AND BENCH
no vocals), but few college participants are likely to prefer clas-
sical music over other forms of music. The emotional implication
of forcing participants to listen to nonpreferred music is not well
understood.
Velten
The original Velten elicitation consisted of self-referent state-
ments designed to be increasingly positive, negative, or neutral.
Participants received instructions to put themselves in the target
mood state and then read each statement for a specified period
(Polivy, 1981; Velten, 1968). Reviews have suggested this tech-
nique effectively elicits positive and negative mood states on
self-report measures (results are less consistent for behavioral
measures; Kenealy, 1986; Larsen & Sinnett, 1991). Studies that
included Velten elicitations as elicitors of mood were not included
in this review. Velten-like elicitations have recently been devel-
oped to elicit discrete emotions, including anger and anxiety, and
studies with the intent to elicit discrete emotions were included in
this review (Carter et al., 2002). The advantages to Velten elici-
tations include the standardized presentation of sentences, al-
though there are variations in the number of statements provided
and the exact instructions given (for a review, see Kenealy, 1986).
As for the other elicitations described thus far, potential disadvan-
tages include the presence of demand characteristics (Clark, 1983;
Polivy & Doyle, 1980) and potential priming of concepts by the
content of the elicitation rather than an evoked emotion (Va¨stfja¨ll,
2002).
Imagination
Emotion elicitation techniques that involve imagination typi-
cally ask participants to listen to or read a scenario while they
imagine themselves in that situation (e.g., Miller, Patrick, & Lev-
enston, 2002; Velasco & Bond, 1998). This technique has the
advantage of allowing participants to draw on their idiosyncratic
experiences while at the same time offering some standardization.
Potential disadvantages include demand effects resulting from the
emotional tone of the imagined scenarios, the possibility that
participants must be willing to place themselves in the emotional
state associated with the fictional scenario, and the risk of priming
content rather than eliciting emotion.
Reading Text
Elicitations that include reading texts consist of participants
reading standardized text intended to elicit an emotional reaction.
The texts range from newspaper clippings describing recent events
to portions from textbooks or novels. Potential advantages include
standardized materials and presentation as well as the fact that
participants are familiar with the media. Here, as with other
elicitations that include specific information, there is the risk that
the elicitations prime cognitive material rather than elicit emotions.
In addition, reading some forms of text, such as an excerpt from a
novel, may require a willingness to engage in a fictional environ-
ment.
Behavioral
Behavioral emotion elicitations encourage participants to be-
have in a particular way with the intention of eliciting discrete
emotions (Laird & Strout, 2007). These elicitations range from
asking people to breathe as if they were experiencing a particular
emotion to encouraging people to activate the same muscles in-
volved in an expression typical of an emotional state (e.g., frown-
ing). Potential advantages to behavioral elicitations include that
emotions are elicited with minimal demand characteristics (de-
pending on how participants are encouraged to alter their behav-
ior), that the elicitation involves participants directly and does not
require a willingness to engage with a fictional scenario, and that
there is little to no cognitive content involved in the elicitation.
Potential disadvantages include the difficulty of standardizing the
elicitation across participants and that these types of elicitations
appear to elicit emotions in only a portion of participants (Laird,
1984; Laird & Crosby, 1974).
Real-Life Manipulations
Real-life manipulations include procedures designed to elicit
emotions through a situation experienced directly by participants.
For example, a confederate may insult a participant in the hallway
outside of a room where outcome measures will be completed
(Eckhardt & Cohen, 1997). These types of elicitations are appeal-
ing because they involve the participant in situations that evoke
emotions in daily life and thus seem intuitively likely to elicit
strong emotions. In addition, if they are executed well, these types
of elicitations are less likely than other elicitations to arouse
participant suspicions or to elicit demand effects from participants
who are not even aware that they are taking part in an emotion
elicitation (Harmon-Jones, Amodio, & Zinner, 2007). Potential
disadvantages with these elicitations include ethical issues related
to the fact that participants cannot be fully informed about the
study beforehand and the difficulty in standardizing the presenta-
tion of a relatively complex task that often includes social inter-
action with a confederate.
Autobiographical Recall
In autobiographical recall elicitations, participants are typically
instructed to recall and often to write about a past event that
elicited intense happiness, sadness, anger, or anxiety or a neutral
event (often an ordinary day or neutral event, such as grocery
shopping; e.g., Lench & Levine, 2005; Lerner & Keltner, 2001).
The time frame in which the recalled event could occur ranges
from days to a lifetime, and the time allotted to recall the event also
varies from a few minutes to unlimited time. Potential advantages
to this elicitation include the lack of any specific cognitive content
and reliance on real events that elicited intense emotions. Potential
disadvantages include that emotional reactions evoked by recalling
past events may not be equivalent to current emotional experience
and that participants must willingly engage in the recollection.
There is also evidence that writing about past emotional events can
lessen the intensity of emotional reactions to those events (Penne-
baker & Chung, 2007), and it is unclear how this might influence
reactions to experimental elicitations.
Key Definitions and the Scope of the Review
Although our purpose in this review was to evaluate evidence of
discrete emotions, it was necessary to develop an a priori definition
837
DISCRETE EMOTION META-ANALYSIS
of emotion in order to identify relevant articles and specify the
scope of the review. There has been growing consensus regarding
the definition of emotion, often through comparison to mood and
affect. Following this consensus, emotion was defined as a rela-
tively brief and intense state in reaction to a specific experience or
event that lasts a few seconds to several minutes (e.g., Clore,
Schwarz, & Conway, 1994; Eich, Kihlstrom, Bower, Forgas, &
Niedenthal, 2000; Ekman, 1992; Rottenberg et al., 2007; Russell,
1991). In contrast, moods and affect were defined as relatively
long-lasting states less tied to a specific experience (Russell,
2003). Emotion, so defined, was the focus of this review, and the
above definitions were used when identifying articles for inclusion
in the review. The review therefore focused on studies that in-
cluded emotion elicited by a specific experience as an independent
variable, and it examined concurrent or subsequent changes in
cognition, judgment, experience, behavior, or physiology.
There is disagreement about which emotions are discrete and
represent independent categories of emotional experience, and,
more fundamentally, there is disagreement about whether emo-
tions represent discrete constructs beyond their valence and
arousal or relation to approach and avoidance motivation. This
review focused on happiness, sadness, anger, and anxiety because
these emotions were elicited in numerous studies, allowing for
comparisons of the effects of different discrete emotions on out-
comes. Inclusion of these emotions also allowed tests of predic-
tions derived from several theories of emotion. A discrete emotion
perspective suggests that each emotion is discrete in the sense that
it has unique causes and consequences. The inclusion of multiple
negative emotions (sadness, anger, anxiety) meant that the possi-
bility that valence and/or arousal, rather than discrete emotions,
accounted for effects could be assessed. A review of potential
discrete positive emotions was not possible because few studies
included more than one of these emotions (Bonanno & Keltner,
2004). Multiple approach emotions (happiness and anger) and
avoidance emotions (sadness and anxiety) were included in order
to assess the possibility that approach or avoidance motivations
might best account for differences among emotions.
In this review, outcome measures were considered to be cogni-
tive, judgment, experiential, behavioral, or physiological out-
comes. Each of these broad outcome categories includes multiple
specific outcomes and measures that could each serve as the focus
of a separate review. We chose to focus on the fundamental issue
of whether emotions elicit changes across systems, and we there-
fore classified outcomes within relatively broad categories. Cog-
nitive outcomes, considered to be measures of thought processes,
included measurements such as recall, attention, and accuracy of
identifying stimuli. Judgment outcomes, considered to be mea-
sures of subjective interpretation, included measurements such as
perceived risk, preference for an interaction partner, and ratings of
the positivity of stimuli. Judgment outcomes were considered
separately from cognitive outcomes, both because judgments are
typically ratings of belief or attitudes, whereas cognitive outcomes
typically indirectly assess thought-related processes, and because
judgments are sometimes considered a behavior. Experiential out-
comes were considered to be self-reports of emotional experience;
they included measurements such as the intensity of experienced
happiness and sadness. Behavioral outcomes were considered to be
observable actions taken by the individual; they included measure-
ments such as facial expression and the amount of hot sauce sent
to an interaction partner. Physiological outcomes were considered
to be physical changes that may or may not be directly observable
without specialized equipment; they included measurements such
as temperature and heart rate (brain and hormone measurements
were excluded because of difficulty interpreting these data and the
exploratory nature of these tests). The outcome categories used in
this review were intentionally broad and were intended to capture
a range of potential reactions associated with emotions.
Potential Methodological Moderators of Emotional
Responses in Experiments
In order to address some of the potential concerns outlined
above for various elicitations and to identify practical issues re-
lated to the elicitation of emotion, we examined several potential
moderators of emotional responses. The age, gender composition,
culture, and population (college students vs. community members)
of participants were included in the present review. Older adults
tend to report less intense negative emotions than younger adults
do after a number of experiences (Carstensen, Fung, & Charles,
2003; Charles & Carstensen, 2008). The average age of partici-
pants may therefore influence elicitation effectiveness, and it was
included as a potential moderator. The proportion of female par-
ticipants may also influence the effectiveness of emotion elicita-
tions, and this potential moderator was included. Women tend to
report more frequent and intense experience of emotion than men
do (Bradley, Codispoti, Sabatinelli, & Lang, 2001). Although
women are more emotionally expressive in general, men are be-
lieved to express certain emotions, particularly anger, more than
women do (e.g., Cox, Stabb, & Bruckner, 1999; Lench, 2006).
This is likely due to the socialization of what is considered appro-
priate within culturally determined gender roles (Grossman &
Wood, 1993) but may reflect evolutionarily adapted changes spe-
cific to gender. Cultural differences are also an important factor in
determining the experience and expression of emotion and were
therefore examined. Cultures vary in the value placed on different
emotions as well as the extent to which there is variation of value
within the culture. Western cultures tend to be more homogeneous
in that there is a consistently high value placed on positive emo-
tions, whereas people within eastern cultures have more varied
experience (Chentsova-Dutton & Tsai, 2007; Eid & Diener, 2001;
Tsai, Levenson, & McCoy, 2006). Due to constraints in the num-
ber of cultures represented in published articles, culture was
roughly approximated by identifying whether participants were
from the United States or other countries. Whether participants
were college students or community members may also be asso-
ciated with the effectiveness of emotion elicitations and therefore
was included. College students often are enrolled in psychology
courses and complete studies in partial fulfillment of course re-
quirements. Student participants may therefore experience demand
characteristics and have insight into study hypotheses to a greater
extent than community members (Sears, 1986).
Although some elicitation techniques appear to have stronger
demand characteristics than others, whether demand effects are
partially responsible for the apparent effects of emotion elicitations
is a broader concern. Cover stories are typically included in order
to make the purpose and predictions of the study less evident and
thus reduce demand effects that might artificially inflate the effect
associated with manipulations. Whether a cover story was offered
838
LENCH, FLORES, AND BENCH
for the emotion elicitation was thus examined. Some articles
included specific cover stories (e.g., that there were two studies,
one regarding recall of past experience and another with the
outcome measure); other studies included sleight of hand that
made the elicitation less obvious (e.g., an apparently unaffiliated
person insulting participants in the hallway). These nuances could
not be efficiently captured through coding; therefore, whether
or not any cover was present was included. In addition, whether or
not the researchers excluded participants on the basis of their
responses to the emotion elicitation was included, because this
exclusion would make elicitations appear more effective than they
were for the entire sample population. The exclusion of partici-
pants ranged from throwing out all women because the elicitation
did not work with them to throwing out a handful of participants
who distracted themselves from the elicitation (e.g., doing push-
ups; walking around the cubicle; Niedenthal, Halberstadt, & Set-
terlund, 1997). Whether participants completed the emotion elic-
itation alone or in a group was also included. Participants may be
more likely to use some emotion regulation strategies, such as
distraction, when in a group. Between- and within-subject com-
parisons were included in the review, and a variable was included
to represent this characteristic. Although the internal validity of
within-subject designs is easily threatened (Shadish, Cook, &
Campbell, 2002), within-subject designs are typical in some areas
of research (e.g., psychophysiological research; Stemmler, Held-
mann, Pauls, & Scherer, 2001). It appeared more comprehensive to
include these areas and account for the contribution relevant to
type of contrast rather than to exclude the research area com-
pletely.
Summary of the Present Investigation
The present investigation was intended to address whether hap-
piness, sadness, anger, and anxiety elicit changes across cognitive,
judgment, experiential, behavioral, and physiological systems;
whether these changes co-occur; and which factors moderate dif-
ferences among discrete emotions. We conducted a meta-analysis
to address these questions by examining effect sizes from com-
parisons among discrete emotions for cognitive, judgment, expe-
riential, behavioral, and physiological outcomes and moderators of
these effects. We also examined the influence of participant age,
gender, country, and population as well as a number of method-
ological factors. The intention was to evaluate the evidence related
to predictions for discrete emotion comparisons that can be drawn
from various theories of emotion and to provide practical infor-
mation about factors that influence emotion elicitation for re-
searchers interested in using emotion as an independent variable.
Method
Selection of Studies
Searches were conducted for potentially relevant studies with
the criteria that articles included the keywords emotion, hap
ⴱ
,
ang
ⴱ
, anx
ⴱ
, or sad
ⴱ
; were not dissertations; and were written in
English. The specific emotion keywords were further limited by
specifying that the article included one of the following keywords:
manip
ⴱ
, elicit
ⴱ
, induct
ⴱ
, or influen
ⴱ
. We reviewed all articles with
the keyword sad
ⴱ
in the year 2009 twice, once using just the
keyword sad
ⴱ
and once adding the manipulation keyword restric
-
tions above. Both searches resulted in the same articles that met
criteria for inclusion. Thus, inclusion of the additional keywords
effectively limited retrieved articles that were unlikely to be in-
cluded in the final review and allowed us to expedite the review
process. Searches were run in PsycInfo from the earliest available
articles (1872) through the end of 2009 (retrieval dates in May and
June, 2010). The relatively broad keywords identified above were
used to allow for the identification of articles that elicited emotion
to examine the effects on other outcomes that were the primary
focus of the investigation. Dissertations were excluded during the
database search because the additional search would have been
prohibitively costly. Only articles written in English were included
because of the language abilities of this review’s authors. Only the
PsycInfo database was searched because additional searches would
have been prohibitively time consuming and because preliminary
searches conducted in MEDLINE and ERIC suggested that very
few studies would meet inclusion criteria (e.g., manipulating mul-
tiple discrete emotions). This search identified 58,280 articles that
potentially met criteria for inclusion in the review.
The titles and then abstracts of these articles were reviewed to
determine their eligibility for the review. This initial review was
extremely inclusive, and articles were excluded at this stage only
if the abstract clearly indicated that the article did not meet criteria
(e.g., if it was evident that the subjects were rats). If there was any
doubt as to the eligibility of the article, the article was retained for
further review. This review of the abstracts identified 1,544 arti-
cles that potentially met inclusion criteria. Each of these articles
was reviewed in full and was coded or excluded because it did not
meet criteria upon this closer examination. A call for additional
papers and unpublished data was made through two e-mail distri-
bution lists relevant to scientists that frequently elicit emotions (the
Society for Personality and Social Psychology and the Interna-
tional Society for Research on Emotion), and direct requests were
made to researchers who frequently use emotion elicitations in
published reports. This call yielded seven additional papers or
descriptions of unpublished data, three of which met criteria for
inclusion.
Articles were excluded based on the following criteria: (a) the
discrete emotions of happiness, sadness, anxiety, or anger were not
examined (articles that dealt with general positive and negative
affect or mood only were not included because they did not fall
within the scope of this review);
1
(b) no comparison groups were
included for happiness, sadness, anger, anxiety, or neutral groups
that were the focus of this review (i.e., only one emotion elicited);
(c) no emotion was elicited; (d) no empirical data were presented
in a format that yielded information about means or effect sizes
(e.g., reviews, qualitative analyses); (e) animals or children were
used as participants (the materials that elicit emotional reactions
may well differ between children or animals and adults; e.g.,
Levine, 1995; Reichenbach & Masters, 1983); or (f) data were
1
Emotion researchers vary in their terminology, and therefore articles
were excluded on the basis of the methodological description rather than
terminology (e.g., a study that focused on “negative affect” would be
included if participants were shown pictures consistent with sadness but
would not be included if participants were shown pictures consistent with
multiple emotions such as disgust, anger, and sadness).
839
DISCRETE EMOTION META-ANALYSIS
redundant with another published source. Participants with severe
psychopathology (e.g., schizophrenia) were excluded, although
these studies typically included a control group that met inclusion
criteria and were therefore included in the review. Participants
with depression and anxiety were included because these are
common disorders and are particularly prevalent in college stu-
dents, the most frequently used population. All analyses remain the
same if these studies are excluded. Participants with subclinical
levels of disorders were also included.
Out of the 1,547 articles reviewed in full (1,544 from database
searches and three from researcher solicitations), 1,037 articles
were excluded for the following reasons: 743 did not elicit
the discrete emotions identified for review, 115 elicited only one
emotion, 76 did not include sufficient information to calculate
effect sizes, 49 reported no empirical data, 35 had no outcome
measure, three had effect sizes that were redundant with another
source, seven used animal or child participants, two were noted by
the journal that published the article as containing faked data, and
seven were not retrievable. In order to assess the reliability of
article identification strategies, three coders (one PhD level and
two master’s level) used the exclusion criteria to identify relevant
articles in a subset of the identified titles (with the keyword sad
ⴱ
in 2009; 59 articles). Each coder worked independently to identify
articles (91%, 92%, and 92%; eight articles were ultimately
included). Overall, 687 studies were identified from 510 sources
that elicited discrete emotions in adults.
Variable Coding
Coding of study variables was completed by two trained coders
(one PhD level and a trained research assistant). Each coder
independently reviewed each full article as many times as neces-
sary to code all study variables. No discussion took place during
this initial coding in order to encourage independent coding of the
study variables. These coders reached the following agreement on
coding (reliability of these codes was assessed with Pearson’s
correlation for continuous variables and a kappa coefficient for
categorical variables): type of elicitation (0.97), emotions
elicited and contrasted (1.00), mean age (r 1.00), proportion
of women (r 0.97), country of participants (1.00), popula-
tion (college student vs. community member, 1.00), presence
of a cover story (0.94), whether participants were excluded
(1.00), and whether participants took part in a group (
0.71). The lead author coded the outcome associated with each
effect size, the type of contrast (between vs. within), and extrap-
olated effect sizes. The coders then met to resolve coding discrep-
ancies. During these meetings, the coders discussed discrepancies
and resolved them by scrutinizing the original source article to
determine which code was consistent with the published text. All
characteristics included in this review were coded according to
statements available in the published texts. Unlike coding that
involves a subjective rating of study characteristics, coding in this
study required little or no subjective judgment. The process of
discussion and consultation of articles therefore resolved all dis-
crepancies (Cooper, Robinson, & Patall, 2006; Pascoe & Richman,
2009).
Outcome characteristics. Discrete emotions are theorized to
elicit changes across systems and to affect the ways that people
think, judge, feel, behave, and react. We therefore sought to
include outcome categories that captured each of these domains.
For this review, outcomes were coded as cognitive outcomes (i.e.,
memory, accuracy, creativity, perception); judgment outcomes
(i.e., judged positivity, preferences, risk judgments; these were
considered separately from cognitive outcomes, because they typ-
ically consist of ratings rather than indirect measures of process
and judgments are sometimes considered behaviors); self-reported
experiential outcomes (i.e., self-reported anger, anxiety, happiness,
sadness, or arousal); behavioral outcomes (i.e., facial expression,
choice, reaction speed, particular action); and physiological out-
comes (i.e., respiration, heart rate, EMG readings, skin conduc-
tance, temperature). Studies that measured physiological outcomes
often included multiple scores that were derived from the same
physiological measurement (e.g., respiratory sinus arrhythmia
scores—RSA—are derived from measures of heart rate and respi-
ration rate, and all three outcomes might be reported). In this case,
only the untransformed measurement was included when available
in order to avoid redundancy (e.g., heart rate and respiration rate).
If the untransformed measurement was not included, redundancy
was not a concern, and the transformed measurement was included
(e.g., RSA).
Elicitation characteristics. The type of elicitation was coded
using the descriptions given above in the introduction. These
included film, picture presentation, priming, music, Velten-like
statements, imagination (this included one virtual reality manipu-
lation), reading text, behavioral, real-life experiences, and autobi-
ographical recall.
Emotions elicited. Researchers vary in their use of emotion
terms, and emotions thus were coded in accordance with the
descriptions outlined in the introduction. Elicitations that involved
the experience of success were coded as happiness elicitations;
those that involved the experience of loss without hope of rein-
stating the goal were coded as sadness elicitations; those that
involved the experience of loss with the possibility of goal rein-
statement were coded as anger elicitations; and those that involved
the experience of threat to goals were coded as anxiety elicitations.
There is disagreement regarding whether certain types of elicita-
tions fulfill these criteria and could be considered to elicit discrete
emotions. For example, researchers have argued that films and
images can elicit emotions such as happiness because viewers
engage with the goals of the protagonists; other researchers might
define positive emotion elicited by film or images as amusement
rather than happiness (Gross & Levenson, 1995). We included in
this review all elicitations that attempted to elicit discrete emotions
and coded them according to the assumption that viewers can
engage in goals of protagonists or with images of nonpresent
stimuli (the effect sizes associated with different elicitations can be
viewed to examine the extent to which such elicitations influence
outcomes). This review focused on anxiety, considered to result
from the perception of potential threats to important goals (e.g.,
contemplation of an upcoming speech), rather than fear, consid-
ered to result from experiencing ongoing threats to important goals
(e.g., one’s chair suddenly dropping backward), in part because
relatively few studies include fear elicitations thus defined (some
studies included elicitations of fear that fit the definition of anxiety
used in this review and were coded accordingly). Similarly, hap-
piness was the only positive emotion included in this review,
because relatively few studies include multiple discrete positive
emotions.
840
LENCH, FLORES, AND BENCH
Methodological moderators. Several potential moderators
of the efficacy of emotion elicitations were coded. Attempts were
made to code several characteristics that may influence the effi-
cacy of emotion elicitations, but this information was available for
only a small subset of studies; therefore, these characteristics could
not be included as potential moderators. These included ethnicity,
the specific outcome measure, the reliability of the outcome mea-
sure, and individual difference characteristics (e.g., depression,
extraversion). The average age of participants and the percentage
of female participants were coded. When the numbers rather than
percentages of male and female participants were reported and it
was clear from the description that a few participants did not report
their gender, the actual percentage of identified female participants
was calculated. The country in which the data were collected (the
United States vs. other countries) was used as a rough proxy for the
culture of participants. If this information was not reported in
the study description, it was assumed that data were collected at
the university with which the first author was affiliated. Whether
participants were college students or community members was
coded. Because college students are the most common conve-
nience sample (Sears, 1986), it was assumed that participants were
college students if no recruitment information was reported in the
study description.
The presence of a cover story for the emotion elicitation was
coded. Cover stories were defined broadly to include any attempt
to disguise the intent of the study or emotion elicitation, ranging
from elaborate deceptions to mildly misleading statements. If no
information was included about a cover story, it was assumed that
no cover was present (in many of these cases, participants were
explicitly instructed to foster the target emotion, and a convincing
cover story would have been difficult to construct). In addition,
whether or not the researchers excluded participants on the basis of
responses to the emotion elicitation was coded. If no mention was
made of participant exclusion, it was assumed that all participants
were included. Whether participants completed the emotion elici-
tation alone or in a group was also coded. It was assumed that
participants in studies that focused on physiological outcomes took
part individually, even if this was not specified, because of the cost
and sensitivity of most monitoring equipment. Participants sepa-
rated by cubicle walls were also coded as taking part individually.
Between- and within-subject comparisons were included in the
review and were coded accordingly. When multiple groups were
available for the same outcome and comparison (e.g., discrete
emotions were compared with a neutral baseline and a neutral
group of participants), the between-subjects comparison was cho-
sen for inclusion. Some studies included both a between- and a
within-subject component in the sense that different groups of
participants were assigned to emotion conditions, but reported
scores represented changes from a neutral baseline (e.g., anger
before an elicitation subtracted from anger after an elicitation in
anger and neutral conditions). If only the difference score was
reported, these studies were coded as between-subjects designs and
the change score was used to calculate the effect size.
Extrapolating and Calculating Effect Sizes
Hedges’ g was used as the effect size because it provides a more
precise estimate of variance than is typically associated with
Cohen’s d (e.g., Hedges & Olkin, 1985; Rosenthal, 1991). For the
purposes of interpreting the magnitude of the observed effects,
effect sizes of .2 are considered small, of .5 are considered me-
dium, and of .8 are considered large. The program Comprehensive
Meta-Analysis (Version 2; CMA; Biostat, 2005) was used to order,
calculate, and compare effect sizes. Effect sizes were calculated
from mean scores on a particular outcome associated with a
discrete emotion (happiness, sadness, anger, and anxiety) com-
pared with mean scores on that same outcome associated with a
different discrete emotion or neutral condition. In studies with a
between-subjects design, the effect size was calculated from mean
scores on an outcome in one emotion condition (e.g., self-reported
anger in the anger condition) compared with mean scores on the
outcome in another emotion condition (e.g., self-reported anger in
the neutral condition). In studies using a within-subject design, the
effect size was calculated from mean scores on an outcome after
one emotion elicitation (e.g., self-reported anger after the anger
elicitation) compared with mean scores on the outcome after a
different emotion elicitation (e.g., self-reported anger after the
same person completed a happy elicitation). When the information
was available in text, table, or figure, effect sizes were calculated
from means and standard deviations. If descriptive statistics were
not available, standard formulas were used to convert inferential
statistics (typically a t test) into effect sizes; if inferential statistics
were unavailable, standard formulas were used to convert associ-
ated p values into effect sizes (for these formulas and procedures,
see Borenstein, Hedges, Higgins, & Rothstein, 2009). If the effect
was reported as nonsignificant but no specific information was
provided, effect sizes were estimated by presuming p .50. If the
effect was reported as significant without additional information,
effect sizes were estimated by presuming p .05, a likely con-
servative estimate because most actual p values would be smaller
and would be associated with a larger effect size (Borenstein et al.,
2009; Rosenthal, 1995).
Effect sizes were calculated for comparisons between neutral,
happiness, sadness, anger, and anxiety conditions involving mea-
sures of cognition, judgment, experience, behavior, or physiology.
This yielded a total of 4,946 effect sizes. When multiple outcomes
or contrasts were assessed within the same study, CMA calculates
the mean effect size and associated combined variance for a study
in order to correct for the dependency of these effects (Borenstein
et al., 2009; Hunter & Schmidt, 2004). For each study, the mean
effect size was calculated as
Y ⫽
1
m
冘
j
m
Y
j
In this formula, m is the number of effects calculated within each
study, Y is the effect size for each outcome or contrast, and j
represents the jth effect calculated within that study. CMA calcu-
lates the combined variance in order to provide statistics related to
the heterogeneity of an effect and inferential analyses to assess
whether the mean effect sizes in two or more groups significantly
differ from one another (similar to an ANOVA). For each study,
the combined variance was calculated as
V
Y
⫽
冉
1
m
冊
2
冉
冘
j1
m
V
i
⫹
冘
jk
r
jk
冑
V
j
冑
V
k
冊
841
DISCRETE EMOTION META-ANALYSIS
In this formula, m is again the number of effects calculated within
each study, V is the variance for each outcome or contrast, and r is
the correlation between the effect sizes. It was typically impossible
to determine the correlations among effect sizes from the original
articles (this would require reports of correlations among scores on
all outcomes or contrasts). Therefore, CMA assumptions were
used, because they result in conservative estimates of homogeneity
for overall effects and comparisons among groups (r 1.00 for
overall effect size calculation; r 0.00 for ANOVA-type analy-
ses; Borenstein et al., 2009).
Random-effects models were calculated because variation in
effect sizes among studies was assumed to occur as the result of
random sampling error as well as differences between groups or
individuals (Cooper & Hedges, 1994). The random effects model
therefore allows generalizations about the effects across a popu-
lation rather than only to past studies (Raudenbush, 1994). Ran-
dom effects models can create small-sample bias (studies that have
small samples and large effects will have a relatively large influ-
ence on the overall effect size estimate; Borenstein et al., 2009).
CMA offers a correction for small-sample bias by weighting the
effect size associated with studies by sample size (Hedges &
Olkin, 1985), and this correction was applied in analyses (N
sample size):
g ⫽ 1 ⫺
3
4N ⫺ 2 ⫺ 1
The Q statistic was used to assess heterogeneity in the variance
among the effect sizes, and, when significant, moderators were
evaluated to explain the heterogeneity. Categorical moderators
were evaluated with the Q
Between
(Q
B
) statistic at the p .05 level
of significance. When significant, Q
B
indicates that the effect sizes
differ among the levels of the categorical moderator (Borenstein et
al., 2009). We evaluated the relationships between continuous
moderators and effect sizes using the meta-regression program
within CMA with a maximum likelihood estimation procedure
(Borenstein et al., 2009; Greenland, 1994; Hedges & Vevea, 1998;
Thompson & Higgins, 2002). With this procedure, the effect size
was regressed on the potential continuous, study-level moderators.
If the slope (b) of the regression line reached the p .05 level of
significance, the variable was considered a moderator of the rela-
tionship between discrete emotion and the outcome variables.
The relationships among changes in different outcome measures
were examined in order to determine if changes across outcomes
co-occurred. To conduct this analysis, we identified articles that
contained more than one outcome measure for a comparison
(number of studies 415; it was not required that a study included
all outcomes but only that it included at least two different out-
comes). The data were transformed to long format in SPSS, such
that each row represented a study and the effect sizes associated
with different outcomes were in separate columns. To account for
potential dependency, we used the effect sizes from the CMA
program calculated during analyses of effects for each outcome for
each study; thus, these effect sizes were weighted and took into
account the dependency of effect sizes. Pearson’s correlation co-
efficients were then calculated between the effect sizes associated
with each outcome and effect sizes of other outcomes to assess the
extent to which changes in the outcomes co-occurred.
Results
Results are presented in six sections. First, descriptive analyses
of general study characteristics are reported. Second, overall dif-
ferences among discrete emotions are considered. Third, the asso-
ciation between discrete emotions and changes in cognitive, expe-
riential, behavioral, and physiological outcomes is assessed, with
separate effect sizes used for each outcome. Fourth, the practical
issue of the effect sizes associated with different types of emotion
elicitations is examined by using type of elicitation as a moderator.
Fifth, characteristics of the sample and study methodological char-
acteristics are examined as potential moderators of the association
between discrete emotions and outcomes. Last, the potential influ-
ence of publication bias is considered.
Descriptive Analyses
Analyses included 510 articles, 687 studies, 4,946 effect sizes,
and 49,473 participants. Table 1 presents a summary of the study
characteristics. As shown, participants tended to be young college
students from the United States. Study samples on average in-
cluded more than 50% women. It should be noted, however, that
effect sizes were frequently based on multiple studies that included
no women (k 257; 6.2%) or no men (k 590; 11.9%), because,
the authors argued, some elicitations have been shown to be more
effective with men or women and it was therefore appropriate to
select participants on the basis of gender. A cover story for the
emotion elicitation was present in over a third of the studies. Few
of the comparisons were from studies that excluded participants on
the basis of their reactions to the emotion elicitation. Less than half
of the comparisons were from studies where participants were in a
group rather than alone and more than half were from studies that
utilized a between-subject designs versus a within-subject design.
As reported in Table 1, the largest number of emotion compar-
isons occurred for happiness versus sadness, followed by happi-
ness or sadness versus neutral conditions. Many of the studies
included self-reported experiential reactions. Film elicitations were
the most common method of emotion elicitation, followed by
imagination, autobiographical recall, and real-life experiences.
Overall Effect Size
The overall effect size associated with comparisons among
discrete emotions was g 0.51, 95% CI [0.48, 0.54], z 34.41,
p .001. Thus, discrete emotions, on average, differed in the
effect they had on outcomes, and this effect would be considered
to be of medium size. There was significant heterogeneity among
these effect sizes, Q(686) 10,748.82, p .001, suggesting that
moderators might account for some of the variance in the effects.
An outlier was identified with effect size values far greater than
other studies (Gross & Levenson, 1995), and this study was
removed from all analyses because of its significant impact on
effect size calculations and comparisons (if included, the average
effect size for film clips is g 1.07).
Two additional composite effect sizes were calculated based on
whether emotion conditions were compared with neutral condi-
tions or with other emotion conditions. These effect size calcula-
tions were conducted at the level of the study and the analyses
selected for the relevant contrasts within each study, resulting in
842
LENCH, FLORES, AND BENCH
effect sizes based only on the relevant contrasts. The overall effect
size associated with comparisons between discrete emotions and
neutral conditions was g 0.46 (460 studies; 95% CI [0.43,
0.49]), z 26.41, p .001, Q(459) 4,649.32, p .001. This
indicates that, on average, discrete emotions had different effects
on outcomes than neutral conditions did. The overall effect size
associated with comparisons among discrete emotions (excluding
contrasts of emotions with neutral conditions) was g 0.58 (443
studies; 95% CI [0.54, 0.62]), z 27.05, p .001, Q(442)
8,061.25, p .001. This indicates that, on average, discrete
emotions had different effects on outcomes than other discrete
emotions did. Thus, effect sizes were medium-sized for emotions
versus neutral conditions and for emotions compared with other
emotions.
Major Theories of Emotion
Effect sizes associated with different outcomes. Is there
evidence that supports the proposition that there are discrete emo-
tions that elicit changes across cognitive, experiential, behavioral,
Table 1
Summary of Study Characteristics and Associated Effect Size Statistics
Characteristic No. of studies M or % (or % of studies) Effect size (g) 95% CI
Overall effect size 687 0.51
ⴱⴱⴱ
[0.48, 0.54]
Participant characteristics
Mean age 259 26.35
% female 539 59.76
From U.S. 687 65%
College students 687 87%
Methodological characteristics
Cover story 687
Present 43% 0.44
ⴱⴱⴱ
[0.44, 0.48]
Absent 0.56
ⴱⴱⴱ
[0.52, 0.61]
Excluded participants 687
Excluded 12% 0.47
ⴱⴱⴱ
[0.39, 0.54]
Did not exclude 0.51
ⴱⴱⴱ
[0.48, 0.55]
Group vs. alone 687
Group 27% 0.46
ⴱⴱⴱ
[0.41, 0.51]
Alone 0.53
ⴱⴱⴱ
[0.49, 0.56]
Between- vs. within-subject 687
Between-subjects 75% 0.48
ⴱⴱⴱ
[0.45, 0.51]
Within-subject 0.59
ⴱⴱⴱ
[0.53, 0.66]
Emotion comparisons
Happiness vs. sadness 346 50% 0.68
ⴱⴱⴱ
[0.62, 0.74]
Happiness vs. anger 59 9% 0.70
ⴱⴱⴱ
[0.56, 0.85]
Happiness vs. anxiety 49 7% 0.96
ⴱⴱⴱ
[0.76, 1.16]
Happiness vs. neutral 229 33% 0.41
ⴱⴱⴱ
[0.36, 0.46]
Sadness vs. anger 63 9% 0.27
ⴱⴱⴱ
[0.19, 0.35]
Sadness vs. anxiety 51 7% 0.26
ⴱⴱⴱ
[0.18, 0.35]
Sadness vs. neutral 263 38% 0.41
ⴱⴱⴱ
[0.36, 0.46]
Anger vs. anxiety 52 8% 0.13
ⴱⴱⴱ
[0.07, 0.19]
Anger vs. neutral 104 15% 0.51
ⴱⴱⴱ
[0.43, 0.60]
Anxiety vs. neutral 113 16% 0.55
ⴱⴱⴱ
[0.45, 0.64]
Outcomes
Cognitive 166 24% 0.24
ⴱⴱⴱ
[0.18, 0.30]
Judgment 234 34% 0.26
ⴱⴱⴱ
[0.22, 0.30]
Self-report experiential 573 83% 0.83
ⴱⴱⴱ
[0.79, 0.88]
Behavioral 156 23% 0.31
ⴱⴱⴱ
[0.25, 0.38]
Physiological 109 16% 0.31
ⴱⴱⴱ
[0.24, 0.38]
Elicitations
Film 162 24% 0.60
ⴱⴱⴱ
[0.53, 0.66]
Pictures 25 4% 0.81
ⴱⴱⴱ
[0.58, 1.03]
Priming 12 2% 0.26
ⴱ
[0.02, 0.49]
Music 49 7% 0.53
ⴱⴱⴱ
[0.41, 0.65]
Velten 93 14% 0.51
ⴱⴱⴱ
[0.44, 0.58]
Imagination 73 11% 0.51
ⴱⴱⴱ
[0.42, 0.61]
Reading text 19 3% 0.41
ⴱⴱⴱ
[0.29, 0.53]
Behavioral 24 3% 0.47
ⴱⴱⴱ
[0.34, 0.60]
Real-life experiences 105 15% 0.46
ⴱⴱⴱ
[0.37, 0.54]
Autobiographical recall 136 20% 0.45
ⴱⴱⴱ
[0.39, 0.51]
Note. Effect sizes are Hedges’ g, with .2 considered small, .5 considered medium, and .8 considered large.
Numbers in brackets are 95% confidence intervals for the effect size. Significance values are associated with the
z statistic.
ⴱ
p .05.
ⴱⴱⴱ
p .001.
843
DISCRETE EMOTION META-ANALYSIS
and physiological outcomes? As already reported, the overall
effect sizes were of medium size, suggesting that there was evi-
dence consistent with the proposition that discrete emotions have
unique effects on outcomes, overall. Effect sizes also differed by
the type of outcome, Q
B
(4) 66.40, p .001. The effect sizes for
each outcome are reported in Table 1. All effects significantly
differed from zero, suggesting that emotions influenced all out-
comes. Self-reported experiential outcomes were associated with a
large effect size, which was larger than the small- to medium-sized
effects associated with cognitive, Q
B
(1) 101.60, p .001;
judgment, Q
B
(1) 95.71, p .001; behavioral, Q
B
(1) 45.32,
p .001; and physiological outcomes, Q
B
(1) 127.72, p .001.
Effect sizes associated with different emotion comparisons.
Type of emotion was a significant moderator of the association
between discrete emotions and the outcomes, Q
B
(9) 114.28, p
.001. The effect sizes associated with each emotion comparison are
reported in Table 1. The effect sizes associated with comparisons
of the influence of happiness versus negative emotions (sadness,
anger, and anxiety) were of moderate to large size. Effect sizes
associated with comparisons of the influence of any emotion
(happiness, sadness, anger, and anxiety) versus neutral conditions
were of moderate size. The effect sizes associated with compari-
sons of the influence of sadness and other negative emotions
(anger, anxiety) were of small to moderate size. The effect size
associated with comparison of the influence of anger versus anx-
iety was quite small, although it significantly differed from zero.
Specific predictions regarding the differences in effect sizes be-
tween discrete emotion comparisons based on several major the-
oretical models of emotion are considered in turn below.
Correlated changes across outcomes. To examine whether
changes in cognition, behavior, experience, and physiology co-
occurred, we identified studies that included more than one out-
come for an emotion comparison (details for this analysis are
presented in the Method section). As reported in Table 2, changes
in self-reported experience were significantly correlated with
changes in behavior and changes in physiology; changes in phys-
iology were significantly correlated with changes in cognition,
changes in behavior, and changes in self-reported experience.
Changes in cognition and changes in judgment did not significantly
correlate with one another or with changes in other outcomes, with the
exception of cognition and physiology. Some correlations were of
small to medium size but were nonsignificant (i.e., the relationship
between judgment and physiology), potentially due to sample size
restrictions. This finding suggests that at least some components of
emotion do covary in response to the elicitation of discrete emotions.
Valence models. Valence models of emotion suggest that
effect sizes should be large between positive and negative emo-
tions and that effect sizes for comparisons among negative emo-
tions should be small, indicating that they have similar effects on
outcomes. To examine this possibility, we coded comparisons as
occurring between positive and negative emotion or between neg-
ative emotions and conducted an analysis to examine whether type
of comparison accounted for variance in the effects. The effect
sizes were indeed larger for comparisons of the influence of
happiness versus negative emotions, g 0.67, 95% CI [0.61,
0.72], z 22.87, p .001, than for comparisons of the influence
of negative emotions versus other negative emotions, g 0.28,
95% CI [0.21, 0.35], z 7.76, p .001, Q
B
(1) 75.53, p .001.
(See Table 1 for specific comparisons; note that this analysis did
not include comparisons to neutral conditions.) However, the
effect size associated with comparisons among negative emotions
was of small to moderate size and significantly differed from zero,
suggesting that valence could not completely account for the
pattern of results.
Some descriptions of valence models suggest that observed
differences among discrete emotions are the result of preconcep-
tions about emotions (i.e., that people experience sadness as dif-
ferent from anger because they have learned to do so). This issue
could not be directly addressed, but, if true, effect sizes should be
larger for self-reported experiential outcomes relative to effect
sizes for other outcomes. As shown in Table 3, effect sizes asso-
ciated with self-reported experience were moderate to large and
the effect sizes for experience were frequently the largest effect
relative to effects associated with all other outcomes. Effect sizes
associated with outcomes other than self-reported experience were
mostly small for contrasts between sadness and anger and anger
and anxiety. It is worth noting that the comparisons between
anxiety versus neutral conditions and anger versus neutral condi-
tions were mostly of moderate size for non-self-report outcomes.
This suggests that the small effect sizes associated with compari-
sons of anger to anxiety or neutral conditions were not due to a
general failure to elicit anger or anxiety. Thus, there was support
for the argument that differences among discrete negative emo-
Table 2
Correlation Coefficients (With Degrees of Freedom) Between Effect Sizes for Different Outcome
Measures Within Studies
Variable 1 2 3 4 5
1. Cognition — .19 (58) .01 (66) .01 (253) .40 (35)
ⴱ
2. Judgment — .14 (63) .08 (292) .27 (25)
3. Behavior — .59 (216)
ⴱⴱⴱ
.28 (54)
ⴱ
4. Experience — .41 (145)
ⴱⴱⴱ
5. Physiology —
Note. To calculate these coefficients, we calculated a combined effect size for each outcome in each study that
included more than one outcome. Correlations were then calculated among those effect sizes. Note that some
small to moderate correlations (i.e., the relationship between changes in judgment and changes in physiology)
are not significant, likely due to small sample size.
ⴱ
p .05.
ⴱⴱⴱ
p .001.
844
LENCH, FLORES, AND BENCH
Table 3
Effect Sizes for Type of Outcome for Each Emotion Comparison
Emotion comparison No. of studies Effect size (g) 95% CI
Happiness vs. sadness
Cognition 90 0.30
ⴱⴱⴱ
[0.21, 0.40]
Judgment 113 0.28
ⴱⴱⴱ
[0.22, 0.35]
Experience 297 1.16
ⴱⴱⴱ
[1.06, 1.26]
Behavior 76 0.51
ⴱⴱⴱ
[0.37, 0.65]
Physiology 34 0.15 [0.10, 0.39]
Happiness vs. anger
Cognition 8 0.24
†
[0.04, 0.72]
Judgment 19 0.34
ⴱ
[0.08, 0.59]
Experience 43 0.95
ⴱⴱⴱ
[0.73, 1.18]
Behavior 10 0.46 [0.19, 1.10]
Physiology 14 0.89
ⴱⴱⴱ
[0.45, 1.34]
Happiness vs. anxiety
Cognition 8 0.53
ⴱ
[0.02, 1.05]
Judgment 8 0.39
†
[0.04, 0.81]
Experience 35 1.61
ⴱⴱⴱ
[1.29, 1.92]
Behavior 6 1.61
ⴱⴱ
[0.42, 2.80]
Physiology 19 0.40
ⴱⴱⴱ
[0.20, 0.61]
Happiness vs. neutral
Cognition 62 0.28
ⴱⴱⴱ
[0.18, 0.38]
Judgment 73 0.21
ⴱⴱⴱ
[0.13, 0.28]
Experience 194 0.70
ⴱⴱⴱ
[0.61, 0.78]
Behavior 51 0.28
ⴱⴱⴱ
[0.16, 0.39]
Physiology 19 0.08 [0.20, 0.36]
Sadness vs. anger
Cognition 7 0.12 [0.14, 0.38]
Judgment 25 0.18
ⴱⴱ
[0.06, 0.31]
Experience 45 0.38
ⴱⴱⴱ
[0.25, 0.50]
Behavior 6 0.18 [0.73, 0.37]
Physiology 14 0.19 [0.05, 0.42]
Sadness vs. anxiety
Cognition 6 0.23 [0.06, 0.53]
Judgment 10 0.34 [0.14, 0.82]
Experience 35 0.34
ⴱⴱⴱ
[0.21, 0.47]
Behavior 4 0.02 [0.88, 0.84]
Physiology 20 0.11
†
[0.01, 0.24]
Sadness vs. neutral
Cognition 72 0.12
ⴱⴱ
[0.03, 0.22]
Judgment 95 0.26
ⴱⴱⴱ
[0.19, 0.33]
Experience 225 0.73
ⴱⴱⴱ
[0.64, 0.82]
Behavior 66 0.15
ⴱⴱ
[0.05, 0.25]
Physiology 17 0.01 [0.15, 0.14]
Anger vs. anxiety
Cognition 4 0.38
ⴱ
[0.07, 0.69]
Judgment 15 0.18 [0.14, 0.18]
Experience 37 0.24
ⴱⴱⴱ
[0.12, 0.37]
Behavior 5 0.20 [0.85, 1.24]
Physiology 27 0.06 [0.05, 0.17]
Anger vs. neutral
Cognition 18 0.36
ⴱⴱⴱ
[0.19, 0.53]
Judgment 31 0.36
ⴱⴱⴱ
[0.20, 0.52]
Experience 76 0.80
ⴱⴱⴱ
[0.61, 0.98]
Behavior 30 0.46
ⴱⴱ
[0.18, 0.74]
Physiology 34 0.55
ⴱⴱⴱ
[0.37, 0.72]
Anxiety vs. neutral
Cognition 25 0.33
ⴱⴱⴱ
[0.15, 0.52]
Judgment 27 0.31
ⴱⴱⴱ
[0.15, 0.46]
Experience 92 1.05
ⴱⴱⴱ
[0.87, 1.24]
Behavior 15 0.27
†
[0.03, 0.57]
Physiology 41 0.33
ⴱⴱ
[0.14, 0.52]
Note. Effect sizes are Hedges’ g, with .2 considered small, .5 considered medium, and .8 considered large.
Some effect sizes are based on a relatively small number of effects and should thus be considered potentially
unreliable. Note that some large effect sizes are not significant due to small sample size or variability associated
with the effect size. Significance values are associated with the z statistic.
†
p .10.
ⴱ
p .05.
ⴱⴱ
p .01.
ⴱⴱⴱ
p .001.
845
DISCRETE EMOTION META-ANALYSIS
tions are greater for outcomes dependent upon conscious aware-
ness than for outcomes less susceptible to conscious biases.
Valence and arousal models. Valence and arousal models
suggest that anger and anxiety should have similar effects on
outcomes. As mentioned, the effect size associated with compar-
isons between anger and anxiety was small relative to that for other
emotion comparisons, although this effect significantly differed
from zero (see Table 1). In contrast to predictions derived from a
valence and arousal model, however, comparisons between sad-
ness and anxiety for non-self-report outcomes were also quite
small; with the exception of cognition, most did not significantly
differ from zero. As noted previously, there did not appear to be a
general failure to elicit anxiety or anger.
Approach and avoidance models. To examine the degree of
support for approach and avoidance models, we combined com-
parisons between emotions of the same motivational direction (i.e.,
happiness vs. anger, sadness vs. anxiety) and combined compari-
sons between emotions of different motivational direction (i.e.,
happiness vs. anxiety, happiness vs. sadness, anger vs. anxiety,
anger vs. sadness). An analysis was conducted to examine whether
type of comparison accounted for variance in the effects. In con-
trast to predictions based on approach and avoidance models, there
was no significant difference between the effect sizes of compar-
isons between emotions of the same motivational direction, g
0.58, 95% CI [0.38, 0.78], z 5.60, p .001, and emotions of
different motivational directions, g 0.60, 95% CI [0.55, 0.65],
z 23.24, p .001), Q
B
(1) 3.03, p .22. The effect sizes
associated with all comparisons on self-report experiential out-
comes were greater than zero (see Table 3 for specific effect sizes),
but effects on other outcomes varied. Some were quite small and
did not differ from zero. As noted, self-reported experience is
particularly likely to be influenced by people’s beliefs about emo-
tions rather than actual differences in emotion. We therefore re-
peated the above analysis excluding self-reported outcomes and
examining only changes in cognitive, judgment, behavioral, and
physiological outcomes. Again, considering only cognitive, judg-
ment, behavioral, and physiological outcomes, there was no sig-
nificant difference between the effect sizes of comparisons be-
tween emotions of the same motivational direction, g 0.43, 95%
CI [0.16, 0.69], z 3.15, p .002, and different motivational
direction, g 0.28, 95% CI [0.24, 0.33], z 12.40, p .001),
Q
B
(1) 1.49, p .48.
Major Types of Elicitations and Related Issues
The type of emotion elicitation moderated the relationship be-
tween discrete emotions and outcomes, Q
B
(9) 26.19, p .003.
As reported in Table 1, the effect sizes associated with all elicita-
tions differed from zero, and most effects were of moderate size.
The effect size associated with priming was smallest, and the effect
size associated with picture elicitations was the largest. Table 4
presents the effect sizes associated with specific elicitation tech-
niques for each discrete emotion comparison. Due to the small
sample sizes in many cases, specific contrasts were not possible.
Films, by far the most frequently used elicitation, were effective
for most emotion comparisons, with the exceptions of anger and
anxiety and sadness and anger. Some elicitations with large effect
sizes were relatively infrequently used, including pictures and
real-life experiences.
Methodological Moderators
A meta-regression analysis using maximum likelihood (de-
scribed in the Method section) revealed that the average age of
participants in the sample did not moderate effect sizes or predict
effect size, b .01, 95% CI [.001, .014], z 1.64, Q
R
2.70,
p .10. It is possible that restricted range influenced this result, as
only 218 out of the nearly 5,000 effect sizes were based on
participants with an average age over 40 years.
A similar meta-regression analysis was conducted to examine
the influence of gender and the proportion of female participants in
a sample did not predict effect size, b .001, 95% CI [.000,
.003], z 1.58, Q
R
2.49, p .11. The social acceptability of
discrete emotions differs for men and women, and the relation of
gender to effect sizes for emotional comparisons that included
anger versus happiness and sadness was therefore examined. Elic-
itations that included happiness and sadness as comparison groups
were more effective as the percentage of women increased, b
.002, 95% CI [.002, .007], z 3.35, Q
R
11.23, p .001, and
this was true both for self-report outcomes, b .006, 95% CI
[.001, .01], z 2.49, Q
R
6.16, p .01, and for non-self-report
outcomes, b .002, 95% CI [.0004, .004], z 2.45, Q
R
6.01,
p .01. In contrast, the proportion of women in the sample did not
predict the effectiveness of elicitations that included anger, b
.001, 95% CI [.003, .001], z 0.86, Q
R
0.74, p .39.
Analyses that included categorical moderators were conducted
comparing the groups of the moderator (as in the overall analysis,
these calculated effect sizes by study to control for dependent
effects). As described previously, most of the studies were con-
ducted in the United States, and country of participants did not
moderate the effect size model, Q
B
(1) 0.02, p .89. Whether
participants were college students or community members also did
not significantly moderate the effect size model, Q
B
(1) 0.92,
p .34.
Means for the effect sizes associated with methodological mod-
erators are shown in Table 1. Effect sizes were significantly
smaller in studies that included a cover story to mask the intent of
the emotion elicitation from participants than in studies that did
not, Q
B
(1) 27.78, p .001. Whether participants were excluded
on the basis of their response to the emotion elicitation did not
significantly influence effect sizes, Q
B
(1) 1.36, p .25; how
-
ever, effect sizes tended to be nonsignificantly somewhat smaller
for studies that excluded participants than for studies that did not
exclude participants. Whether study participants took part as a
group or as individuals was a marginal moderator of effect sizes,
Q
B
(1) 5.38, p .06. Effect sizes tended to be larger when
participants took part individually rather than in groups. Effect
sizes were influenced by whether studies utilized a within- or
between-subjects design, Q
B
(1) 9.28, p .01. Effect sizes from
studies that used within-subject comparisons were larger than
those from studies that used between-subjects comparisons.
Evaluation of Publication Bias
A funnel plot, trim-and-fill method, and fail-safe ns were ex-
amined to evaluate the likelihood that publication bias (the ten-
dency for published studies available for meta-analyses to report
significant results) influenced the results of this review. A funnel
plot is a scatter plot showing the effect size in relation to sample
846
LENCH, FLORES, AND BENCH
Table 4
Effect Sizes for Type of Emotion Elicitation for Each Emotion Comparison
Emotion comparison No. of studies Effect size (g) 95% CI
Happiness vs. sadness
Film 106 0.88
ⴱⴱⴱ
[0.76, 1.00]
Pictures 15 1.02
ⴱⴱⴱ
[0.52, 1.53]
Prime 4 0.07 [0.31, 0.17]
Music 39 0.66
ⴱⴱⴱ
[0.49, 0.85]
Velten 48 0.74
ⴱⴱⴱ
[0.62, 0.86]
Imagine 26 0.72
ⴱⴱⴱ
[0.47, 0.97]
Read text 7 0.47
ⴱⴱⴱ
[0.27, 0.66]
Behavior 11 0.48
†
[0.07, 1.02]
Real 19 0.54
ⴱⴱⴱ
[0.42, 0.66]
Recall 75 0.49
ⴱⴱⴱ
[0.41, 0.57]
Happiness vs. anger
Film 14 0.87
ⴱⴱⴱ
[0.53, 1.22]
Pictures 2 0.17 [0.39, 0.73]
Prime 1 0.22 [0.29, 0.72]
Music 2 1.16
ⴱⴱⴱ
[0.71, 1.62]
Velten 3 0.26
†
[0.02, 0.53]
Imagine 10 0.55
ⴱⴱⴱ
[0.25, 0.84]
Read text 3 0.25 [0.36, 0.86]
Behavior 9 1.03
ⴱⴱⴱ
[0.55, 1.51]
Real 5 0.94
ⴱⴱⴱ
[0.66, 1.21]
Recall 11 0.59
ⴱⴱⴱ
[0.27, 0.91]
Happiness vs. anxiety
Film 15 1.08
ⴱⴱⴱ
[0.78, 1.40]
Pictures 5 2.04
ⴱⴱ
[0.78, 3.31]
Prime 2 0.06 [0.55, 0.67]
Music 6 1.87
ⴱⴱ
[0.74, 3.00]
Velten 0
Imagine 9 1.08
ⴱⴱⴱ
[0.53, 1.63]
Read text 0
Behavior 7 0.68
ⴱⴱ
[0.25, 1.12]
Real 1 0.10 [0.39, 0.59]
Recall 5 0.77
ⴱⴱⴱ
[0.34, 1.21]
Happiness vs. neutral
Film 88 0.45
ⴱⴱⴱ
[0.38, 0.53]
Pictures 8 0.47
ⴱⴱⴱ
[0.20, 0.73]
Prime 5 0.05 [0.26, 0.16]
Music 11 0.30
ⴱⴱ
[0.13, 0.47]
Velten 34 0.45
ⴱⴱⴱ
[0.34, 0.56]
Imagine 24 0.48
ⴱⴱⴱ
[0.33, 0.63]
Read text 7 0.56
ⴱⴱⴱ
[0.27, 0.86]
Behavior 3 0.37
ⴱ
[0.07, 0.67]
Real 19 0.36
ⴱⴱⴱ
[0.20, 0.52]
Recall 31 0.26
ⴱⴱⴱ
[0.16, 0.35]
Sadness vs. anger
Film 13 0.16
†
[0.01, 0.33]
Pictures 0
Prime 0
Music 2 0.30
ⴱⴱⴱ
[0.19, 0.41]
Velten 3 0.10 [0.22, 0.41]
Imagine 8 0.18
ⴱⴱ
[0.05, 0.32]
Read text 8 0.41
ⴱⴱⴱ
[0.25, 0.57]
Behavior 9 0.21
ⴱⴱ
[0.05, 0.38]
Real 3 0.19 [0.92, 0.52]
Recall 19 0.36
ⴱⴱ
[0.14, 0.58]
Sadness vs. anxiety
Film 14 0.22
ⴱⴱ
[0.08, 0.35]
Pictures 4 0.43
†
[0.05, 0.90]
Prime 0
Music 6 0.47 [0.10, 21.07]
Velten 6 0.23
ⴱ
[0.01, 0.44]
Imagine 10 0.30
ⴱⴱ
[0.11, 0.49]
Read text 0
Behavior 8 0.18
†
[0.01, 0.37]
Real 0
Recall 6 0.40 [0.09, 0.89]
(table continues)
847
DISCRETE EMOTION META-ANALYSIS
size. It is examined visually for symmetry (asymmetry indicates a
potential publication bias because the size of the sample is related
to the effect). A funnel plot we constructed appeared asymmetri-
cal, as studies in the bottom half of the plot fell on the right of the
mean effect size. This suggested that there was evidence of bias, as
smaller studies were associated with larger effect sizes. It is
important to note that funnel plots reveal bias, but this bias may
reflect actual differences among studies rather than publication
bias (Borenstein et al., 2009). These “small-study effects” occur
when studies with small samples also tend to use methodologies
associated with larger effects (Sterne & Egger, 2001). In the
present review, studies that assessed the effects of emotion on
physiological outcomes tended to use small samples and tended to
use within-subject designs. Physiological outcomes and within-
subject designs were both associated with large effect sizes. There-
fore, the asymmetry in the funnel plot may indicate the effects of
these physiological studies rather than the effects of publication
bias.
In order to assess the potential impact of the missing studies on
the overall effect size, we used the Duval and Tweedie (2000)
trim-and-fill technique to impute the effect sizes associated with
the potential missing studies. The funnel plot with these imputed
values is presented in Figure 1. As can be seen, this technique
revealed an observed effect size of .37 and an adjusted effect size
of .26, suggesting that the missing studies that might result from
publication bias would have little impact on the overall effect size
Table 4 (continued)
Emotion comparison No. of studies Effect size (g) 95% CI
Sadness vs. neutral
Film 93 0.50
ⴱⴱⴱ
[0.40, 0.59]
Pictures 8 0.35
ⴱⴱⴱ
[0.17, 0.53]
Prime 2 0.16 [0.43, 0.11]
Music 15 0.28
ⴱⴱⴱ
[0.16, 0.41]
Velten 61 0.36
ⴱⴱⴱ
[0.28, 0.44]
Imagine 27 0.41
ⴱⴱⴱ
[0.19, 0.63]
Read text 8 0.43
ⴱⴱⴱ
[0.28, 0.58]
Behavior 2 0.38 [0.71, 0.04]
Real 13 0.39
ⴱⴱⴱ
[0.27, 0.52]
Recall 37 0.37
ⴱⴱⴱ
[0.23, 0.51]
Anger vs. anxiety
Film 10 0.05 [0.13, 0.03]
Pictures 2 0.02 [0.22, 0.19]
Prime 2 0.51
ⴱⴱⴱ
[0.26, 0.76]
Music 2 0.34 [0.65, 1.33]
Velten 1 0.08 [0.68, 0.51]
Imagine 13 0.24
ⴱⴱⴱ
[0.11, 0.37]
Read text 1 0.37
ⴱⴱⴱ
[0.24, 0.50]
Behavior 8 0.12
ⴱ
[0.06, 0.54]
Real 6 0.03 [0.36, 0.72]
Recall 8 0.05 [0.11, 0.20]
Anger vs. neutral
Film 14 0.30
ⴱⴱ
[0.13, 0.48]
Pictures 2 0.72 [0.96, 2.40]
Prime 4 0.26
†
[0.04, 0.55]
Music 1 0.18 [0.06, 0.42]
Velten 1 0.21 [0.82, 0.40]
Imagine 19 0.56
ⴱⴱⴱ
[0.38, 0.74]
Read text 3 0.72
ⴱ
[0.17, 1.27]
Behavior 3 0.96
ⴱ
[0.14, 1.78]
Real 35 0.57
ⴱⴱⴱ
[0.42, 0.72]
Recall 24 0.52
ⴱⴱⴱ
[0.32, 0.71]
Anxiety vs. neutral
Film 17 0.51
ⴱⴱⴱ
[0.30, 0.73]
Pictures 8 1.61
ⴱⴱⴱ
[0.85, 2.36]
Prime 6 0.56
ⴱⴱⴱ
[0.26, 0.85]
Music 4 0.31
ⴱⴱ
[0.12, 0.51]
Velten 5 0.27 [0.13, 0.67]
Imagine 25 0.64
ⴱⴱⴱ
[0.37, 0.91]
Read text 2 0.09 [0.04, 0.22]
Behavior 2 0.42 [0.74, 1.58]
Real 35 0.47
ⴱⴱⴱ
[0.32, 0.63]
Recall 12 0.55
ⴱⴱⴱ
[0.28, 0.82]
Note. Effect sizes are Hedges’ g, with .2 considered small, .5 considered medium, and .8 considered large.
Some effect sizes are based on a relatively small number of effects and should thus be considered potentially
unreliable. Note that some medium to large effect sizes are not significant due to small sample size or variability
associated with the effect size. Significance values are associated with the z statistic.
†
p .10.
ⴱ
p .05.
ⴱⴱ
p .01.
ⴱⴱⴱ
p .001.
848
LENCH, FLORES, AND BENCH
and that the effect was robust. Additional analyses also suggested
that the overall effect was robust and was unlikely to be an artifact
of publication bias. A classic fail-safe n calculation suggested that
2,612 studies with nonsignificant findings would have to exist in
order for the reported effect to be nonsignificant (Rosenthal,
1991). Orwin’s fail-safe n, a more conservative calculation, indi-
cated that the mean effect size in studies outside the report would
have to be .00000 in order for the reported effects to be nonsig-
nificant. We believe that concern over publication bias was further
reduced in this particular review, because studies often reported
significant effects in the predicted direction for some study vari-
ables and not for others and we included all comparisons in this
review. For example, a particular elicitation for sadness might
result in more self-reported sadness (as predicted) but also more
self-reported anger (in contrast to predictions).
Discussion
The present investigation was intended to synthesize past re-
search in which emotions were elicited as an experimental manip-
ulation. Our purpose in the review was to address whether happi-
ness, sadness, anger, and anxiety were associated with differences
in cognition, judgment, experience, behavior, and physiology;
whether these changes co-occur; and which factors moderate these
effects. A review and research synthesis was conducted of existing
studies in order to address these questions. The overall purpose
was to provide information about emotion and emotion elicitations
that would be of theoretical and practical value.
Major Theories of Emotion
There is growing agreement that emotions are evolutionarily
adaptive responses that serve to organize cognitive, judgment,
experiential, behavioral, and physiological reactions to changes in
the environment (e.g., Ekman, 1992; Frijda, 1987; Izard, 1971;
Lerner & Keltner, 2001; Mauss et al., 2005; Pinker, 1997; Rotten-
berg et al., 2007). One of our primary goals in the present review
was to evaluate the extent to which the existing research evidence
was consistent with this view.
Discrete emotion models. Overall, there was a medium
effect size across all comparisons among the discrete emotions
included in this review. There was also a medium effect size for all
emotions compared with neutral conditions and all emotions com-
pared with other emotions. Further, the effect sizes associated with
all of the comparisons among discrete emotions were greater than
zero, although the effect sizes associated with the comparison
between anger and anxiety and anger and neutral conditions were
quite small. The small difference between anger and anxiety may
be due to an underlying similarity, perhaps on an arousal dimen-
sion, as suggested by valence and arousal accounts of emotion. It
is also possible, however, that emotion elicitations have not effec-
tively targeted one emotion (e.g., a film segment that shows a bully
picking on a child may elicit anger and anxiety). The effect sizes
associated with the comparison between discrete emotions and
neutral conditions were all significantly greater than zero, suggest-
ing that emotional experiences were indeed elicited in experimen-
tal settings. There also was evidence that behavioral, experiential,
and physiological responses tended to covary following elicitation
of the discrete emotions of happiness, sadness, anger, and anxiety.
Cognition and judgment did not correlate with one another or with
other changes. Theories of emotion vary in the stress placed on
various components of emotion. For example, some theories stress
the cognitive and judgment changes that occur with the experience
of discrete emotions (e.g., Lerner & Keltner, 2001), whereas other
theories stress the co-occurrence of experiential, physiological,
0150-501-
0.0
0.5
1.0
1.5
2.0
Standard Error
Hedges's g
Funnel Plot of Standard Error by Hedges's g
Figure 1. The funnel plot presents the standard error on the y-axis and the Hedges’ g effect size on the x-axis
(the bottom half of the plot represents smaller studies). White circles represent observed data points and black
circles represent imputed data points from trim-and-fill procedures that take the place of possible studies that
may be missing due to publication bias. On the x-axis, the white diamond (top mark) represents the mean effect
size without potential publication bias taken into account; the black diamond (bottom mark) represents the mean
effect size with the imputed values.
849
DISCRETE EMOTION META-ANALYSIS
and facial expression behaviors (e.g., Izard, 1971). Theories that
stress components essential to emotion (behavior, physiology, and
experience) may receive better empirical support than theories that
focus on components that are primed by emotion but are not
necessarily integral (cognition, judgment). Effects associated with
these less integral components may be more likely to be moderated
by individual and situational factors.
In general, the findings are consistent with those of studies that
have shown changes across systems following emotional experi-
ence (Frijda, 1987; Lench & Levine, 2005; Lerner & Keltner,
2001; Mauss et al., 2005). There were differences, however, in the
effect sizes associated with different comparisons among discrete
emotions, and these differences may be informative about the
extent to which the existing evidence is consistent with different
theories of emotion. It is important to remember when evaluating
these findings that the inferences are limited by the fact that some
proposed discrete emotions were not represented in this review
(e.g., pride, disgust) and that we examined only studies that elic-
ited discrete emotions and did not include studies that manipulated
valence or arousal only. It is possible that some emotion models
might be more clearly supported with emotions other than those
examined in this review or in studies whose manipulations are
based on those models. And, of course, the ability to draw infer-
ences from a pattern of findings in the literature is limited by the
methodological rigor of the studies included in the review.
Valence models. Valence and valence plus arousal have been
posited to capture important aspects of emotional experience (e.g.,
Barrett, 1998; Russell, 1980). Effect sizes associated with com-
parisons that contrasted happiness and negative emotions were
indeed larger than effect sizes associated with comparisons among
negative emotions, as would be predicted from valence models.
However, the effect size associated with the latter comparisons
was greater than zero and would be considered small in size. This
finding suggests that there are small but consistent differences in
the effects of the discrete negative emotions of sadness, anger, and
anxiety that cannot be captured by valence models.
Valence and arousal models. Most models describe anger
and anxiety as similar in terms of valence (negative) and arousal
(high), and this comparison was therefore examined for evidence
of the importance of valence and arousal (Feldman-Barrett &
Russell, 1998; Larsen & Diener, 1992; Russell, 1980; Watson &
Tellegen, 1985). In the present review, the effect size associated
with the comparison between anger and anxiety was indeed a small
effect size relative to those for other emotion comparisons. These
findings suggest that a valence and arousal model captures impor-
tant information about the differences among emotions. But the
effect size associated with the comparison between sadness and
anxiety was also small, and this relatively small effect cannot be
accounted for in valence and arousal models that consider sadness
a low-arousal emotion and anxiety a high-arousal emotion. Thus,
there was evidence that valence and arousal capture important
information about the differences among emotions, but there re-
mained differences in outcomes among discrete emotions that
could not be explained by these models. A portion of this differ-
ence was captured by including whether outcomes were self-
reported or not, but some of the findings remained inconsistent
with a valence and arousal model even after the type of outcome
was considered. It is important to remember in evaluating this
evidence that the review included only studies that attempted to
elicit discrete emotions. Thus, it cannot be informative about the
effect sizes associated with elicitations of general positive or
negative states.
Some theorists who support a valence or valence/arousal model
of emotion have suggested that differences in discrete emotions
result from culturally developed expectations or beliefs about
emotion (e.g., Barrett & Russell, 1999). In the present investiga-
tion, the effect sizes associated with comparisons among all emo-
tions were consistently greater than zero only for self-reported
experiences. For measures other than self-report, the effect sizes
that compared anxiety and anger, sadness and anger, and sadness
and anxiety were small. Although this finding does not clearly
support a valence model, it does suggest that differences among
discrete emotions of the same valence may be—particularly for
negative emotions compared with anxiety—the result of beliefs
about emotions. This possibility remains to be directly tested.
Anxiety is notoriously difficult to elicit through standard emotion
elicitations (e.g., Gross & Levenson, 1995), but this difficulty
cannot explain why anxiety elicitations would effectively elicit
changes in self-reported outcomes relative to other negative emo-
tions but not other outcomes. Nor can it explain the finding that the
effect size associated with anxiety compared with neutral groups
was greater than zero for self-report and for other outcomes. Thus,
there was no evidence that anxiety influenced outcomes other than
self-report in ways that differed from other negative emotions.
Approach and avoidance models. Other theories of emotion
stress the degree to which emotions are associated with approach
or avoidance motivations (Bechara et al., 1997; Carver, 2004;
Gray, 1972). Approach and avoidance models that describe posi-
tive emotions as approach emotions and negative emotions as
avoidance emotions are supported by the same evidence des-
cribed above for valence models. Other models suggest that hap-
piness and anger are approach emotions and sadness and anxiety
are avoidance emotions (Bodenhausen et al., 1994; Harmon-Jones,
2003; Henriques & Davidson, 1991). The effect sizes associated
with emotions of different motivational directions were no greater
than those associated with emotions of the same motivational
direction, which is inconsistent with an approach and avoidance
model. When only outcomes that did not rely on self-report were
examined, the findings offered mixed support for an approach and
avoidance model. Multiple comparisons were consistent with pre-
dictions from these models, and other comparisons were inconsis-
tent with predictions.
Major Types of Elicitations and Related Issues
The present meta-analysis was also designed to answer practical
questions about the effectiveness of experimental emotion elicita-
tions. Researchers often have little or no guidance about the type
of elicitation method to select and methodological issues that may
impact its efficacy. The majority of elicitations appeared to elicit
the target emotions to a greater extent than neutral conditions or
other nontarget emotions. Velten-type elicitations have been
shown to effectively elicit general positive and negative states
(Gerrards-Hesse et al., 1994; Westermann et al., 1996); however,
these elicitations were not especially effective elicitors of discrete
emotions in the present review. There was no evidence that per-
sonally experienced elicitations, such as real-life experiences or
recalled experiences, resulted in larger effect sizes than did those
850
LENCH, FLORES, AND BENCH
that were not personally experienced, such as imagined scenarios
or reading text. Indeed, the presentation of pictures appeared to be
a particularly effective elicitor of discrete emotions, although it
was infrequently used. Film was the most common method of
emotion elicitation, and it appeared just as effective as other
elicitation techniques for most emotion comparisons. Several elic-
itations were associated with large effect sizes but were infre-
quently used, including pictures and real-life experiences. These
may be useful elicitations in future research, particularly for emo-
tional states that have proven difficult to elicit in an experimental
context (e.g., anger and anxiety).
Potential Methodological Moderators
There was no evidence that age influenced the degree of differ-
ences among discrete emotions (e.g., Carstensen et al., 2003;
Charles & Carstensen, 2008); however, most participants were
young. A clear test of the proposition that age influences the
intensity of discrete emotional experiences would require addi-
tional studies that include older participants. Elicitations of hap-
piness and sadness were more effective as the percentage of
women in the sample increased, and this was true for self-reported
experiential outcomes and other outcomes; elicitations of anger
and anxiety were not related to the proportion of women in the
sample. This finding suggests that gender may influence the in-
tensity of some discrete emotional experiences, consistent with
arguments that the acceptability of emotional experiences or
adapted physiology for men or women may dictate how people
interpret and react to situations (e.g., Cox et al., 1999; Lench,
2006). Culture and college student savvy have been proposed as
potential moderators of the intensity of discrete emotional experi-
ences (e.g., Sears, 1986; Tsai et al., 2006), but there was no
evidence of these effects in the present review. This is consistent
with the view that emotional experiences are universal; however,
most participants were U.S. college students, and lack of variabil-
ity might have been responsible for the null findings. Further,
culture was only very roughly approximated by comparing studies
in the United States to those in all other countries, and a more
nuanced approach might reveal cultural differences. It is also
important to note that few studies included information about the
ethnic composition of the sample, and this potential moderator
could not be included in the review. This information is critical for
future studies on discrete emotions and may help reveal cultural
differences in emotional reactions. The gender differences coupled
with lack of cultural differences might suggest that only individual
characteristics with a strong biological basis, such as gender, may
influence emotional reactions. However, more work is needed that
includes measures related to culture or participants from various
cultures before such a conclusion can be drawn (Henrich, Heine, &
Norenzayan, 2010).
About half of the studies incorporated a cover story to mitigate
the demand characteristics inherent in emotion elicitations. This is
potentially problematic because studies that included a cover story
had significantly smaller effect sizes than studies that did not
include a cover story, suggesting that a substantial portion of
effects of discrete emotions may be due to demand characteristics
in the studies. To the extent that researchers are interested in
capturing the effects of emotions rather than participants’ beliefs
about the effects of emotions, it is likely important to include a
cover story. Whether participants were excluded on the basis of
their response to the emotion elicitation also tended to moderate
the effectiveness of elicitations. Effect sizes tended to be smaller in
studies that excluded participants than in those that did not exclude
participants. It is possible that the exclusion of some participants
may have been sufficient to result in a marginal significance value
reaching the traditional cutoff for significance tests (p .05). In
order to identify effective techniques and the effects of emotion,
studies should include all participants in analyses, regardless of
whether they appeared to be impacted by the elicitation. Emotion
elicitations tended to be more effective when participants took part
as individuals (i.e., alone in a room or separated by cubicle walls)
rather than in groups. This suggests that there may be some
diffusion of emotional intensity that occurs during group elicita-
tions, possibly because participants are distracting or comforting
themselves by attending to other people.
Portrait of an Effective Discrete Emotion Elicitation
What does an effective emotion elicitation look like? It uses
films, pictures, music, or imagination to elicit emotion. The choice
among these effective elicitations must be made based on the
theoretical interests of the researcher and practical constraints (the
References and online supplemental materials identify studies in
which the various types of emotion elicitations were used). Films,
pictures, and imagined events are likely to prime specific cognitive
content, and researchers interested in the effects of emotion inde-
pendent of cognitive content may want to avoid these elicitations.
Behavior, real-life events, and recalled events also prime cognitive
content, but if participants are randomly assigned to conditions one
can assume that specific cognitive content should be distributed
among conditions and should not constitute a threat to validity.
Whether standardization within or between conditions is most
important should also be considered when choosing an elicitation.
It is relatively easy to standardize the presentation and content of
films, pictures, and music within a condition, as the researcher can
control how the stimuli are viewed, how long they are viewed, the
specific images, and surrounding circumstances (Kring & Gordon,
1998; Rottenberg et al., 2007). In addition, there are standardized
sets of these elicitations that are available to researchers. It is
difficult to standardize across these conditions, however, as the
stimuli typically vary on multiple dimensions, including the social
aspects of the images, instruments included in music, and the color
of images, all of which may influence emotional reactions. In
contrast, it is relatively easy to standardize behavior and real-life
event elicitations across conditions in the sense that the same event
can be enacted with only one crucial difference designed to elicit
the target emotion. But these elicitations are difficult to standardize
within conditions because of experimenter and participant vari-
ability.
The discrete emotions of interest and practical constraints may
also determine researchers’ choice of elicitations. Previous re-
search has suggested that anxiety may be particularly difficult to
elicit in a laboratory setting (e.g., Gross & Levenson, 1995; Larsen
& Ketelaar, 1991). It is possible that this difficulty is in part due to
the fact that many elicitations require the suspension of disbelief
and participants’ willing engagement with stimuli (e.g., films,
pictures, music, behavior). Real-life experiences may be more
useful in eliciting anxiety, and there is evidence that these elicita-
851
DISCRETE EMOTION META-ANALYSIS
tions elicit a large physiological stress response (for a review, see
Dickerson & Kemeny, 2004). Some elicitations may also be im-
possible due to practical constraints. For example, structuring
real-life experiences often requires multiple experimenters and
confederates, and this may not be possible without teams of
research assistants.
Methodological choices should also be governed by theoretical
interests and practical constraints. The gender makeup of the
sample must also be determined by practical constraints and by the
interests of the researcher, but it should be borne in mind that some
emotions will be more effectively elicited depending on the gender
composition of the sample. In studies focused on general and
theoretically universal constructs or relationships, it would be
critical to include men and women and to assess for gender
differences. Additional research related to the reasons for gender
differences in reactions to emotion elicitations would also be
informative about the nature of emotions and their relation to
cultural expectations and beliefs.
Although the inclusion of a cover story was associated with
smaller effect sizes, it is important in most cases that studies
include a cover story. Without a cover it is impossible to determine
if any effects are the result of emotions or of demand effects
related to participants’ beliefs about emotions and beliefs about the
experimenter’s expectations. The majority of researchers are in-
terested in the effects of emotions on outcomes. Therefore, the
inclusion of a cover story is critical, as is avoiding explicit demand
statements that participants should simulate emotions.
To heighten the efficacy of elicitations, participants should be in
separate rooms or at least separated by cubicle walls. Group
elicitations are more efficient in the sense that data can be col-
lected relatively quickly from large groups of participants; how-
ever, there appears to be a cost in terms of the associated effect for
group elicitations. The effectiveness of some elicitations that might
work quite well individually may be reduced in groups, and this
may lead researchers to conclude that the manipulation was inef-
fective.
Limitations and Recommendations for Future Studies
This review is based on reported studies that included almost
5,000 effect sizes and nearly 50,000 participants, yet some of the
comparisons were based on relatively few available effect sizes. In
some cases this limited the ability to draw conclusions, particularly
in reference to the effectiveness of priming elicitations and the
influence of age, culture, and college student status on experimen-
tal emotion elicitations. More studies are required that include
these variables with diverse samples in order to broaden inferences
about the effectiveness of elicitations. Relative to happiness and
sadness, anger and anxiety were included in few studies. The
present review suggests that these emotions have separable effects
on at least some outcomes and that additional studies that elicit
multiple discrete emotions are required in order to fully understand
the implications of discrete emotions for processing and behavior.
Further, as very few studies included multiple discrete positive
emotions, they could not be included in the review. Additional
work is needed in order to determine if discrete positive emotions
have differing effects. Nearly half of the effect sizes were for
self-reported emotional reactions. Although self-reports provide
important information about emotional experience, theory suggests
that discrete emotions should influence cognition, behavior, and
physiology. Research that includes multiple outcomes will test this
proposal. Additional research is also needed about how quickly
emotions fade after an experience. This information is of practical
importance for researchers designing studies on the effects of
emotions and is of theoretical importance, given that emotions are
defined as relatively brief and intense experiences following an
event. An attempt was made in the present review to code the time
between emotion elicitation and measurements, but few studies
provided this information. It is worth noting that many unusually
large effect sizes (though not statistical outliers) were associated
with physiological outcomes. Recent critiques have suggested that
some physiological measures, particularly electrophysiology and
neuroimaging, may have increased potential for Type I error
(Kriegeskorte, Simmons, Bellgowan, & Baker, 2009; Vul, Harris,
Winkielman, & Pashler, 2009). A similar issue may exist in
methodologies that assess changes in autonomic nervous system
activity. The cause of this concern is the practice of selectively
analyzing a subset of data based on researcher identification of
relevant data points; with autonomic nervous system data, this
includes identifying time periods to be analyzed and removing
perceived artifact data points (Kriegeskorte et al., 2009; Mendes,
2009).
Our conclusions in this review are also limited by the exclusion
criteria we used to identify relevant studies. Studies that attempted
to elicit discrete emotions were included, and studies that exam-
ined moods or general positive and negative states were not
included. Therefore, the inferences that can be made are relevant
only to discrete emotions. If this review had included studies that
manipulated emotional states in a manner consistent with valence
or approach/avoidance models, this may have increased the em-
pirical support garnered for these models, because the published
results would be more likely to be consistent with such models.
The question addressed by this review was not the degree of
support for these models but rather the degree to which evidence
related to discrete emotions was congruent with predictions de-
rived from these models.
Conclusions
The study of emotion has taken a prominent place in several
psychological theories, and the present review suggests that place-
ment is well deserved, as emotions change the way people think,
feel, and behave. Our intent in this review was to examine evi-
dence for discrete emotions and to provide practical information
on effective emotion elicitations. Although there are definitely
gaps in our knowledge regarding ways to classify emotional re-
sponses and reliably elicit responses in an experimental context,
this review addresses some of the fundamental questions by ex-
amining numerous studies conducted by researchers across areas.
In order for the study of emotion to continue to thrive, it is critical
that researchers arrive at some agreement regarding what emotion
is and what it does.
References
References with an asterisk were included in the meta-analysis synthesis
and are discussed in the text. These references are accompanied by super-
scripts following the asterisk to indicate the type (or types) of emotion
852
LENCH, FLORES, AND BENCH
elicitation used in that article (B behavior, D read text, F film, I
imagine, L recall, M music, P pictures, Pr prime, R real life,
V Velten). For a complete list, go to http://dx.doi.org/10.1037/
a0024244.supp
*
M,V
Albersnagle, F. A. (1988). Velten and musical mood induction pro
-
cedures: A comparison with accessibility in thought associations. Be-
haviour Research and Therapy, 26, 79 –96. doi:10.1016/0005-
7967(88)90035-6
Bargh, J. A., & Gollwitzer, P. M. (1994). Environmental control of goal-
directed action: Automatic and strategic contingencies between situa-
tions and behavior. In W. D. Spaulding (Ed.), Nebraska Symposium on
Motivation: Vol. 41. Integrative views of motivation, cognition, and
emotion (pp. 77–124).
Barrett, L. F. (1998). Discrete emotions or dimensions? The role of valence
focus and arousal focus. Cognition & Emotion, 12, 579 –599. doi:
10.1080/026999398379574
Barrett, L. F., & Russell, J. A. (1999). The structure of current affect:
Controversies and emerging consensus. Current Directions in Psycho-
logical Science, 8, 10–14. doi:10.1111/1467-8721.00003
Bechara, A., Damasio, H., Tranel, D., & Damasio, A. R. (1997, February
28). Deciding advantageously before knowing the advantageous strat-
egy. Science, 275, 1293–1294. doi:10.1126/science.275.5304.1293
Biostat. (2005). Comprehensive meta-analysis [Computer software]. Re-
trieved from http://www.meta-analysis.com
Bodenhausen, G. V., Mussweiler, T., Gabriel, S., & Moreno, K. N. (2001).
Affective influences on stereotyping and intergroup relations. In J. P.
Forgas (Ed.), Handbook of affect and social cognition (pp. 319 –343).
Mahwah, NJ: Erlbaum.
*
B,L
Bodenhausen, G. V., Sheppard, L. A., & Kramer, G. P. (1994).
Negative affect and social judgment: The differential impact of anger
and sadness. European Journal of Social Psychology, 24, 45– 62. doi:
10.1002/ejsp.2420240104
Bonanno, G. A., & Keltner, D. (2004). The coherence of emotion systems:
Comparing “on-line” measures of appraisal and facial expressions, and
self-report. Cognition & Emotion, 18, 431– 444. doi:10.1080/
02699930341000149
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009).
Introduction to meta-analysis. Chichester, England: Wiley.
Bower, G. H., & Forgas, J. P. (2000). Affect, memory, and social cogni-
tion. In E. Eich, J. F. Kihlstrom, G. H. Bower, J. P. Forgas, & P. M.
Niedenthal (Eds.), Cognition and emotion (pp. 87–168). New York, NY:
Oxford University Press.
Bradley, M. M., Codispoti, M., Sabatinelli, D., & Lang, P. J. (2001).
Emotion and motivation II: Sex differences in picture processing. Emo-
tion, 1, 300–319. doi:10.1037//1528-3524.1.3.300
Bradley, M. M., & Lang, P. J. (2000). Affective reactions to acoustic
stimuli. Psychophysiology, 37, 204 –215. doi:10.1017/
S0048577200990012
Bradley, M. M., & Lang, P. J. (2007). The International Affective Picture
System (IAPS) in the study of emotion and attention. In J. A. Coan &
J. J. B. Allen (Eds.), Handbook of emotion elicitation and assessment
(pp. 29–46). New York, NY: Oxford University Press.
Brown, G. W. (2000). Emotion and clinical depression: An environmental
view. In M. Lewis & J. M. Haviland-Jones (Eds.), Handbook of emotion
(pp. 75–90). New York, NY: Guilford Press.
Carstensen, L. L., Fung, H. H., & Charles, S. T. (2003). Socioemotional
selectivity theory and the regulation of emotion in the second half of life.
Motivation and Emotion, 27, 103–123. doi:10.1023/A:1024569803230
*
V
Carter, L. E., McNeil, D. W., Vowles, K. E., Sorrell, J. T., Turk, C. L.,
Ries, B. J., & Hopko, D. R. (2002). Effects of emotion on pain reports,
tolerance, and physiology. Pain Research and Management, 7, 21–30.
Retrieved from http://www.pulsus.com
Carver, C. S. (2004). Negative affects deriving from the behavioral ap-
proach system. Emotion, 4, 3–22. doi:10.1037/1528-3542.4.1.3
Center for the Study of Emotion and Attention. (1995). The International
Affective Picture System: Digitized photographs. Gainesville: University
of Florida, Center for Research in Psychophysiology.
Charles, S. T., & Carstensen, L. L. (2008). Unpleasant situations elicit
different emotional responses in younger and older adults. Psychology
and Aging, 23, 495–504. doi:10.1037/a0013284
Chentsova-Dutton, Y. E., & Tsai, J. L. (2007). Gender differences in
emotional response among European Americans and Hmong Americans.
Cognition & Emotion, 21, 162–181. doi:10.1080/02699930600911333
Clark, D. M. (1983). On the induction of depressed mood in the laboratory:
Evaluation and comparison of the Velten and musical procedures. Ad-
vances in Behaviour Research and Therapy, 5, 27– 49. doi:10.1016/
0146-6402(83)90014-0
Clore, G. L., Schwarz, N., & Conway, M. (1994). Affective causes and
consequences of social information processing. In R. S. Wyer & T. K.
Srull (Eds.), Handbook of social cognition (pp. 323–417). Hillsdale, NJ:
Erlbaum.
Cooper, H., & Hedges, L. V. (1994). The handbook of research synthesis.
New York, NY: Russell Sage Foundation.
Cooper, H., Robinson, J. C., & Patall, E. A. (2006). Does homework
improve academic achievement? A synthesis of research 1987–2003.
Review of Educational Research, 76, 1– 62. doi:10.3102/
00346543076001001
Cox, D. L., Stabb, S. D., & Bruckner, K. H. (1999). Women’s anger:
Clinical and developmental perspectives. Ann Arbor, MI: Braun-
Brumfield.
Dickerson, S. S., & Kemeny, M. E. (2004). Acute stressors and cortisol
responses: A theoretical integration and synthesis of laboratory research.
Psychological Bulletin, 130, 355–391. doi:10.1037/0033-
2909.130.3.355
Duval, S., & Tweedie, R. (2000). A nonparametric trim and fill method of
accounting for publication bias in meta-analysis. Journal of the Ameri-
can Statistical Association, 95, 89–98.
*
R
Eckhardt, C. I., & Cohen, D. J. (1997). Attention to anger-relevant and
irrelevant stimuli following naturalistic insult. Personality and Individ-
ual Differences, 23, 619– 629. doi:10.1016/S0191-8869(97)00074-3
Eich, E., Kihlstrom, J. F., Bower, G. H., Forgas, J. P., & Niedenthal, P. M.
(Eds.). (2000). Cognition and emotion. New York, NY: Oxford Univer-
sity Press.
Eid, M., & Diener, E. (2001). Norms for experiencing emotions in different
cultures: Inter- and intranational differences. Journal of Personality and
Social Psychology, 81, 869– 885. doi:10.1037//0022-3514.81.5.869
Ekman, P. (1992). Are there basic emotions? Psychological Review, 99,
550 –553. doi:10.1037/0033-295X.99.3.550
Epstein, S. (1994). Integration of the cognitive and the psychodynamic
unconscious. American Psychologist, 49, 709–724. doi:10.1037/0003-
066X.49.8.709
Feldman-Barrett, L., & Russell, J. A. (1998). Independence and bipolarity
in the structure of current affect. Journal of Personality and Social
Psychology, 74, 967–984. doi:10.1037/0022-3514.74.4.967
Fredrickson, B. L. (2001). The role of positive emotions in positive
psychology: The broaden-and-build theory of positive emotions. Amer-
ican Psychologist, 56, 218–226. doi:10.1037/0003-066X.56.3.218
Frijda, N. H. (1987). Emotion, cognitive structure, and action tendency.
Cognition & Emotion, 1, 115–143. doi:10.1080/02699938708408043
Gerrards-Hesse, A., Spies, K., & Hesse, F. W. (1994). Experimental
inductions of emotional states and their effectiveness: A review. British
Journal of Psychology, 85, 55–78.
Gray, J. A. (1972). The psychophysiological basis of introversion extra-
version: A modification of Eysenck’s theory. In V. D. Nebylitsyn & J. A.
Gray (Eds.), The biological basis of individual behavior (pp. 182–205).
San Diego, CA: Academic Press.
*
I
Green, J. D., & Sedikides, C. (1999). Affect and self-focused attention
853
DISCRETE EMOTION META-ANALYSIS
revisited: The role of affect organization. Personality and Social Psy-
chology Bulletin, 25, 104–119. doi:10.1177/0146167299025001009
*
R
Green, R. A., & Murray, E. J. (1975). Expression of feeling and cogni
-
tive reinterpretation in the reduction of hostile aggression. Journal of
Consulting and Clinical Psychology, 43, 375–383. doi:10.1037/
h0076701
Greenland, S. (1994). A critical look at some popular meta-analytic meth-
ods. American Journal of Epidemiology, 140, 290–296. Retrieved from
http://aje.oxfordjournals.org
Gross, J. J. (1998). Antecedent- and response-focused emotion regulation:
Divergent consequences for experience, expression, and physiology.
Journal of Personality and Social Psychology, 74, 224 –237. doi:
10.1037/0022-3514.74.1.224
Gross, J. J., & Levenson, R. W. (1995). Emotion elicitation using films.
Cognition & Emotion, 9, 87–108. doi:10.1080/02699939508408966
Grossman, M., & Wood, W. (1993). Gender differences in intensity of
emotional experience: A social role interpretation. Journal of Personal-
ity and Social Psychology, 65, 1010 –1022. doi:10.1037/0022-
3514.65.5.1010
Haidt, J. (2001). The emotional dog and its rational tail: A social intuition-
ist approach to moral judgment. Psychological Review, 108, 814–834.
doi:10.1037/0033-295X.108.4.814
Harmon-Jones, E. (2003). Anger and the behavioral approach system.
Personality and Individual Differences, 35, 995–1005. doi:10.1016/
S0191-8869(02)00313-6
Harmon-Jones, E., Amodio, D. M., & Zinner, L. R. (2007). Social psy-
chological methods of emotion elicitation. In J. A. Coan & J. J. B. Allen
(Eds.), Handbook of emotion elicitation and assessment (pp. 91–105).
New York, NY: Oxford University Press.
*
B
Harmon-Jones, E., & Sigelman, J. (2001). State anger and prefrontal
brain activity: Evidence that insult-related relative left-prefrontal acti-
vation is associated with experienced anger and aggression. Journal of
Personality and Social Psychology, 80, 797– 803. doi:10.1037/0022-
3514.80.5.797
Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis.
Orlando, FL: Academic Press.
Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in
meta-analysis. Psychological Methods, 3, 486 –504. doi;10.1037/1082-
989X.3.4.486
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in
the world? Behavioral and Brain Sciences, 33, 61– 83. doi:10.1017/
S0140525X0999152X
Henriques, J. B., & Davidson, R. J. (1991). Left frontal hypoactivation in
depression. Journal of Abnormal Psychology, 100, 535–545. doi:
10.1037/0021-843X.100.4.535
Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting
error and bias in research findings (2nd ed.). New York, NY: Sage.
Isen, A. M. (2000). Positive affect and decision making. In M. Lewis &
J. M. Haviland-Jones (Eds.), Handbook of emotion (pp. 417– 435). New
York, NY: Guilford Press.
Izard, C. E. (1971). The face of emotion. East Norwalk, CT: Appleton-
Century-Crofts.
Izard, C. E., & Abe, J. A. (2004). Developmental changes in facial
expressions of emotions in the strange situation during the second year
of life. Emotion, 4, 251–265. doi:10.1037/1528-3542.4.251
Izard, C. E., Hembree, E. A., & Huebner, R. R. (1987). Infants’ emotion
expressions to acute pain: Developmental change and stability of indi-
vidual differences. Developmental Psychology, 23, 105–113. doi:
10.1037/0012-1649.23.1.105
Izard, C. E., Levinson, K. L., Ackerman, B. P., Kogos, J. L., & Blumberg,
S. H. (1998). Children’s emotional memories: An analysis in terms of
differential emotions theory. Imagination, Cognition and Personality,
18, 173–188. Retrieved from http://www.baywood.metapress.com
Kahneman, D. (2003). A perspective on judgment and choice: Mapping
bounded rationality. American Psychologist, 58, 697–720. doi:10.1037/
0003-066X.58.9.697
Kenealy, P. (1988). Validation of a music mood induction procedure: Some
preliminary findings. Cognition & Emotion, 2, 41– 48. doi:10.1080/
02699938808415228
Kenealy, P. M. (1986). The Velten mood induction procedure: A method-
ological review. Motivation and Emotion, 10, 315–335. doi:10.1007/
BF00992107
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Baker, C. I.
(2009). Circular analysis in systems neuroscience: The dangers of dou-
ble dipping. Nature Neuroscience, 5, 535–540. doi:10.1038/nn.2303
*
F
Kring, A. M., & Gordon, A. H. (1998). Sex differences in emotion:
Expression, experience, and physiology. Journal of Personality and
Social Psychology, 74, 686–703. doi:10.1037/0022-3514.74.3.686
Laird, J. D. (1984). The real role of facial response in the experience of
emotion: A reply to Tourangeau and Ellsworth, and others. Journal of
Personality and Social Psychology, 47, 909 –917. doi:10.1037/0022-
3514.47.4.909
Laird, J. D., & Crosby, M. (1974). Individual differences in self-attribution of
emotion. In H. London & R. Nisbett (Eds.), Thinking and feeling: The
cognitive alteration of feeling states (pp. 213–234). Newbury Park, CA:
Sage.
Laird, J. D., & Strout, S. (2007). Emotional behaviors as emotional stimuli.
In J. A. Coan & J. J. B. Allen (Eds.), Handbook of emotion elicitation
and assessment (pp. 54 – 64). New York, NY: Oxford University Press.
Lang, P. J. (1995). The emotion probe: Studies of motivation and emotion.
American Psychologist, 50, 372–385. doi:10.1037/0003-066X.50.5.372
Larsen, R. J., & Diener, E. (1992). Promises and problems with the circumplex
model of emotion. In M. S. Clark (Ed.), Emotion: Review of personality and
social psychology (pp. 25–59). Thousand Oaks, CA: Sage.
Larsen, R. J., & Ketelaar, T. (1991). Personality and susceptibility to
positive and negative emotional states. Journal of Personality and Social
Psychology, 61, 132–140. doi:10.1037/0022-3514.61.1.132
Larsen, R. J., & Sinnett, L. M. (1991). Meta-analysis of experimental
manipulations: Some factors affecting the Velten mood induction pro-
cedure. Personality and Social Psychology, 17, 323–334. doi:10.1177/
0146167291173013
Lench, H. C. (2006). Anger disorders: Diagnosing unrecognized mood
disorders. In T. Plante (Ed.), Mental disorders of the new millennium
(pp. 53–72). Westport, CT: Praeger.
*
L
Lench, H. C., & Levine, L. J. (2005). Effects of fear on risk and control
judgments and memory: Implications for health promotion messages. Cog-
nition & Emotion, 19, 1049 –1069. doi:10.1080/02699930500203112
Lerner, J. S., & Keltner, D. (2000). Beyond valence: Toward a model of
emotion-specific influences on judgement and choice. Cognition &
Emotion, 14, 473–493. doi:10.1080/026999300402763
*
L
Lerner, J. S., & Keltner, D. (2001). Fear, anger, and risk. Journal of
Personality and Social Psychology, 81, 146 –159. doi:10.1037/0022-
3514.81.1.146
Levine, L. J. (1995). Young children’s understanding of the causes of anger
and sadness. Child Development, 66, 697–709. doi:10.2307/1131944
Loewenstein, G. (1996). Out of control: Visceral influences on behavior.
Organizational Behavior and Human Decision Processes, 65, 272–292.
doi:10.1006/obhd.1996.0028
Mauss, I. B., Levenson, R. W., McCarter, L., Wilhelm, F. H., & Gross, J. J.
(2005). The tie that binds? Coherence among emotion experience, be-
havior, and physiology. Emotion, 5, 175–190. doi:10.1037/1528-
3542.5.2.175
*
I
Mayer, J. D., Gayle, M., Meehan, M. E., & Haarman, A. K. (1990).
Towards better specification of the mood-congruency effect in recall.
Journal of Experimental Social Psychology, 26, 465–480. doi:10.1016/
0022-1031(90)90051-M
Mendes, W. B. (2009). Assessing autonomic nervous system activity. In E.
854
LENCH, FLORES, AND BENCH
Harmon-Jones & J. S. Beer (Eds.), Methods in social neuroscience (pp.
118 –147). New York, NY: Guilford Press.
Mikels, J. A., Fredrickson, B. L., Larkin, G. R., Lindberg, C. M., Maglio,
S. J., & Reuter-Lorenz, P. A. (2005). Emotional category data on images
from the International Affective Picture System. Behavior Research
Methods, 37, 626 – 630. Retrieved from http://brm.psychonomic-
journals.org
*
I
Miller, M. W., Patrick, C. J., & Levenston, G. K. (2002). Affective
imagery and the startle response: Probing mechanisms of modulation
during pleasant scenes, personal experiences, and discrete negative
emotions. Psychophysiology, 39, 519 –529. doi:10.1017/
S0048577202394095
*
M
Niedenthal, P. M., Halberstadt, J. B., & Setterlund, M. B. (1997). Being
happy and seeing “happy”: Emotional state mediates visual word rec-
ognition. Cognition & Emotion, 11, 403– 432. doi:10.1080/
026999397379863
Pascoe, E. A., & Richman, L. S. (2009). Perceived discrimination and
health: A meta-analytic review. Psychological Bulletin, 135, 531–554.
doi:10.1037/a0016059
Pennebaker, J. W., & Chung, C. K. (2007). Expressive writing, emotional
upheavals, and health. In H. S. Friedman & R. C. Silver (Eds.), Foun-
dations of health psychology (pp. 263–284). New York, NY: Oxford
University Press.
*
F
Philippot, P. (1993). Inducing and assessing differentiated emotion–
feeling states in the laboratory. Cognition & Emotion, 7, 171–193.
doi:10.1080/02699939308409183
Pinker, S. (1997). How the mind works. New York, NY: Norton.
Plutchik, R. (2000). Emotions in the practice of psychotherapy: Clinical
implications of affect theories. Washington, DC: American Psycholog-
ical Association.
*
R,V
Polivy, J. (1981). On the induction of emotion in the laboratory:
Discrete moods or multiple affect states? Journal of Personality and
Social Psychology, 41, 803–817. doi:10.1037/0022-3514.41.4.803
Polivy, J., & Doyle, C. (1980). Laboratory induction of mood states
through the reading of self-referent mood statements: Affective changes
or demand characteristics? Journal of Abnormal Psychology, 89, 286 –
290. doi:10.1037/0021-843X.89.2.286
Raudenbush, S. W. (1994). Random effects model. In H. Cooper & L. V.
Hedges (Eds.), The handbook of research synthesis (pp. 301–321). New
York, NY: Russell Sage Foundation.
Reichenbach, L., & Masters, J. (1983). Children’s use of expressive and
contextual cues in judgments of emotion. Child Development, 54, 993–
1004. doi:10.2307/1129903
Robinson, M. D., & Clore, G. L. (2002). Belief and feeling: Evidence for
an accessibility model of emotional self-report. Psychological Bulletin,
128, 934–960. doi:10.1037/0033-2909.128.6.934
Rosenthal, R. (1991). Meta-analytic procedures for social research. Thou-
sand Oaks, CA: Sage.
Rosenthal, R. (1995). Writing meta-analytic reviews. Psychological Bul-
letin, 118, 183–192. doi:10.1037/0033-2909.118.2.183
Rottenberg, J., Ray, R. D., & Gross, J. J. (2007). Emotion elicitation using
film. In J. A. Coan & J. B. Allen (Eds.), Handbook of emotion elicitation
and assessment (pp. 9–28). New York, NY: Oxford University Press.
Rozin, P. (1999). The process of moralization. Psychological Science, 10,
218 –221. doi:10.1111/1467-9280.00139
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality
and Social Psychology, 39, 1161–1178. doi:10.1037/h0077714
Russell, J. A. (1991). Culture and the categorization of emotions. Psycho-
logical Bulletin, 110, 426– 450. doi:10.1037/0033-2909.110.3.426
Russell, J. A. (2003). Core affect and the psychological constructs of
emotion. Psychological Review, 110, 145–172. doi:10.1037/0033-
295X.110.1.145
Scherer, K., & Zentner, M. (2008). Music evoked emotions are different:
More often aesthetic than utilitarian. Behavioral and Brain Sciences, 31,
595–596. doi:10.1017/S0140525X08005505
Sears, D. O. (1986). College sophomores in the laboratory: Influences of a
narrow data base on social psychology’s view of human nature. Journal
of Personality and Social Psychology, 51, 515–530. doi:10.1037/0022-
3514.51.3.515
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and
quasi-experimental designs for generalized causal inference. Boston,
MA: Houghton Mifflin.
Shiller, V. M., Izard, C. E., & Hembree, E. A. (1986). Patterns of emotion
expression during separation in the strange-situation procedure. Devel-
opmental Psychology, 22, 378–382. doi:10.1037/0012-1649.22.3.378
Silvia, P. J. (2005). Emotional responses to art: From collation and arousal
to cognition and emotion. Review of General Psychology, 9, 342–357.
doi:10.1037/1089-2680.9.4.342
*
R
Stemmler, G., Heldmann, M., Pauls, C. A., & Scherer, T. (2001).
Constraints for emotion specificity in fear and anger: The context counts.
Psychophysiology, 38, 275–291. doi:10.1017/S0048577201991668
Sterne, J. A., & Egger, M. (2001). Funnel plots for detecting bias in
meta-analysis: Guidelines on choice of axis. Journal of Clinical Epide-
miology, 54, 1046–1055. Retrieved from http://www.ncbi.nlm.nih.gov/
pubmed/11576817
Thompson, S., & Higgins, J. (2002). How should meta-regression analyses
be undertaken and interpreted? Statistics in Medicine, 21, 1559 –1573.
Tomkins, S. S., & McCarter, R. (1964). What and where are the primary
affects? Some evidence for a theory. Perceptual and Motor Skills, 18,
119 –158.
Tsai, J. L., Levenson, R. W., & McCoy, K. (2006). Cultural and temper-
amental variations in emotional response. Emotion, 6, 484–497. doi:
10.1037/1528-3542.6.3.484
Va¨stfja¨ll, D. (2002). Emotion induction through music: A review of the
musical mood induction procedure. Musicae Scientiae, 6, 173–211.
*
I
Velasco, C., & Bond, A. (1998). Personal relevance is an important
dimension for visceral reactivity in emotional imagery. Cognition &
Emotion, 12, 231–242. doi:10.1080/026999398379736
Velten, E. (1968). A laboratory task for induction of mood states. Behav-
iour Research and Therapy, 6, 473– 482. doi:10.1016/0005-
7967(68)90028-4
Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009). Puzzlingly high
correlations in fMRI studies of emotion, personality, and social cogni-
tion. Perspectives on Psychological Science, 4, 274 –290. doi:10.1111/
j.1745-6924.2009.01125.x
Walters, K. S. (1989). The law of apparent reality and aesthetic emotions.
American Psychologist, 44, 1545–1546. doi:10.1037/0003-
066X.44.12.1545
Watson, D., & Tellegen, A. (1985). Toward a consensual structure of
mood. Psychological Bulletin, 98, 219 –235. doi:10.1037/0033-
2909.98.2.219
Weinstein, J., Averill, J. R., Opton, E. M., & Lazarus, R. S. (1968).
Defensive style and discrepancy between self-report and physiological
indexes of stress. Journal of Personality and Social Psychology, 10,
406 –413. doi:10.1037/h0026829
Westermann, R., Spies, K., Stahl, G., & Hesse, F. (1996). Relative effec-
tiveness and validity of mood induction procedures: A meta-analysis.
European Journal of Social Psychology, 26, 557–580.
doi:10.1002/(SICI)1099-0992(199607)26:4557::AID-EJSP7693.0
.CO;2-4
Received November 10, 2009
Revision received April 23, 2011
Accepted May 5, 2011 䡲
855
DISCRETE EMOTION META-ANALYSIS
A preview of this full-text is provided by American Psychological Association.
Content available from Psychological Bulletin
This content is subject to copyright. Terms and conditions apply.
