Insect reward systems: Comparing flies and bees

Abstract

Many elements of animal behaviour are organised by an innate reward-seeking drive stemming from neurobiological reward systems. The behavioural concept of reward and its neurobiological substrates was initially developed in mammalian systems, and there it has become clear that several novel social behaviours evolved through the co-option of reward pathways. Only more recently has reward been explored in insects. In this review, we consider current knowledge about reward pathways in the two predominant insect models: Drosophila melanogaster and the honey bee Apis mellifera. These two models are phylogenetically distantly related and have vastly different ecologies: fruit flies are mostly solitary while honey bees live in complex societies involving social foraging and brood care. Initially, it was assumed the reward system was essentially similar between these two organisms, but more recent studies have appeared to highlight quite significant differences. Here, we critically evaluate apparent differences in the neurobiology of the reward system between these organisms. We discuss which differences may be real and which may be reflective of the very different modes of analysis applied in these two models. Finally, we discuss how modification of reward systems might have contributed to social evolution in insects.

Full text

CHAPTER SIX
Insect Reward Systems:
Comparing Flies and Bees
Eirik Søvik*
,1
, Clint J. Perry
†
, Andrew B. Barron
{
*Department of Biology, Washington University in St. Louis, St. Louis, Missouri, USA
†
School of Chemical and Biological Sciences, Queen Mary University, London, United Kingdom
{
Department of Biological Sciences, Macquarie University, Sydney, New South Wales, Australia
1
Corresponding author: e-mail address: eirik.sovik@gmail.com
Contents
1. Introduction 190
2. What Do We Mean by Reward? 191
3. Reward Systems in Honey Bees and Flies 194
3.1 Experimental approaches 194
3.2 Neuroanatomy of insect reward systems 197
3.3 Octopamine signals reward in insects 203
3.4 Dopamine in insect reward 208
3.5 Modulation of reward seeking 211
3.6 Non-food rewards 212
3.7 Summary of findings from honey bees and fruit flies 212
4. Reward Systems and Social Evolution 213
5. Concluding Remarks 217
Acknowledgements 218
References 218
Abstract
Many elements of animal behaviour are organised by an innate reward-seeking drive
stemming from neurobiological reward systems. The behavioural concept of reward
and its neurobiological substrates was initially developed in mammalian systems,
and there it has become clear that several novel social behaviours evolved through
the co-option of reward pathways. Only more recently has reward been explored in
insects. In this review, we consider current knowledge about reward pathways in the
two predominant insect models: Drosophila melanogaster and the honey bee Apis
mellifera. These two models are phylogenetically distantly related and have vastly dif-
ferent ecologies: fruit flies are mostly solitary while honey bees live in complex societies
involving social foraging and brood care. Initially, it was assumed the reward system was
essentially similar between these two organisms, but more recent studies have
appeared to highlight quite significant differences. Here, we critically evaluate apparent
differences in the neurobiology of the reward system between these organisms. We
discuss which differences may be real and which may be reflective of the very different
Advances in Insect Physiology, Volume 48 #2015 Elsevier Ltd
ISSN 0065-2806 All rights reserved.
http://dx.doi.org/10.1016/bs.aiip.2014.12.006
189

modes of analysis applied in these two models. Finally, we discuss how modification of
reward systems might have contributed to social evolution in insects.
1. INTRODUCTION
In behavioural neuroscience, reward has an operational definition;
stimuli that elicit approach or seeking behaviour from animals (e.g. sugars
or other nutrients) are considered rewards, and the attainment of these
stimuli is considered to be rewarding (Loeb, 1918). Reward and its neuro-
biological correlates have been the focus of study for almost a century (Loeb,
1918), but until the 1990s the majority of this work was conducted in
mammalian model systems (Schultz, 2010). This rich research tradition
(predominantly with mammals) has resulted in a nuanced understanding
of the neural substrates that are involved in signalling different conceptual
aspects of reward (Berridge et al., 2009), and an understanding of how
reward pathways can become the driving force of the evolution of novel
social behaviours (O’Connell and Hofmann, 2011). However, this domi-
nant focus on mammalian systems has done little to advance our knowledge
of how reward systems operate in other animals, such as the insects.
The study of reward systems in insects was inspired by a landmark study
(Hammer, 1993), demonstrating that the activation of a single identified
neuron could replace sucrose reward during conditioning in honey bees
(Apis mellifera). Since this pivotal study, a range of studies have examined dif-
ferent aspects of rewards in insects and have attempted to pinpoint the neural
substrates responsible (Perry and Barron, 2013). From these studies, there
appeared to be quite significant differences between the results from insects
and the more established understanding of the physiology of reward in
mammals. In mammals, dopamine (DA) systems emerged as key neuro-
chemical signals for rewards (Schultz, 1997, 2002, 2007), while in insects
octopamine (OA) systems were considered to signal rewards (Hammer
and Menzel, 1998). OA and DA are chemically quite similar but OA is only
a trace amine in mammalian nervous systems.
Recently, this view has changed following a number of studies using the
fruit fly (Drosophila melanogaster) that have demonstrated a role for specific
DA circuits in the fly reward system (Burke et al., 2012; Kim et al.,
2007; Krashes et al., 2009; Selcho et al., 2009). However, so far no evidence
has emerged that DA plays a role in reward signalling in honey bees. Some
190 Eirik Søvik et al.

have suggested that this reflects neurobiological differences between the
reward systems of honey bees and fruit flies, while others have suggested that
the difference might reflect differences in the experimental methods applied
to these two organisms (Barron et al., 2010; Perry and Barron, 2013). In this
review, we carefully compare and contrast the current status of knowledge
in the two systems. We consider the real and apparent differences between
these two influential models. Finally, we speculate how modification of the
reward system by the evolutionary process might have contributed to the
evolution of social behaviour in insects.
2. WHAT DO WE MEAN BY REWARD?
The idea that behaviour is guided by reward and punishment goes
back at least to the ancient Greeks (e.g. Epicurus, translated in Bailey,
1926). Reward and reward seeking is now recognised to be fundamental
to the organisation of behaviour (Dreher and Tremblay, 2009). Reward
not only supports elementary processes such as drinking, eating and repro-
duction but also encompasses a broad range of behaviour from arousal and
attention to foraging and decision making to gambling and social relation-
ships. Dysfunctions in the reward system can cause disruptions in learning,
planning, social interactions and general evaluation of our surroundings
(Søvik and Barron, 2013). Understanding the reward system has become
vital for understanding all goal-directed animal behaviour, from the very
basic to the most complex. Understandably, the focus of research has been
on mammalian models, and here we briefly review how analyses of reward
in mammals have helped develop our understanding of biological reward
systems.
What do we mean by reward? One of the earliest, and perhaps still most
useful, experimental definitions of reward began with Loeb (1918) who
defined rewards as stimuli that elicit approach and punishment as stimuli that
elicit avoidance. Pavlov (1927) conceptualised rewards as objects that bring
about a change in behaviour, or rather that caused learning. These defini-
tions are operational in that the behaviour that the stimulus induces qualifies
it as a reward or punishment. In common usage, reward is often synonymous
with the word pleasure. In general writing and many sociological disciplines,
it is common to consider anything that gives us the conscious experience of
pleasure as a reward. Scientific investigation of reward, however, has shown
that reward involves multiple mechanisms and psychological components.
191Insect Reward Systems: Comparing Flies and Bees

The major components of reward include liking: the conscious and uncon-
scious pleasurable impact of reward; wanting: the conscious and unconscious
desires for reward; and learning: associations, representations and predictions
about future rewards based on past experiences (Berridge and Kringelbach,
2008). Extensive research in mammalian and human systems has demon-
strated that these different psychological components are mediated by partly
dissociable brain substrates (Berridge et al., 2009).
Direct brain stimulation, functional neuroimaging, neural recording and
tracing studies have identified several overlapping areas in the mammalian
brain that respond to reward, including areas within the brain stem and mid-
brain up into the frontal cortex, with the strongest response to reward
occurring along the medial forebrain bundle (Leknes and Tracey, 2008;
Wise and Rompre, 1989). Lesion and psychopharmacological studies sug-
gest that the midbrain DA systems are vital to reward processing in the
mammalian brain (Schultz, 2010). Early studies on electrical self-stimulation
and drug addiction showed that stimulation of midbrain areas with a strong
dopaminergic innervation would elicit approach behaviour, and dopami-
nergic projections within the midbrain play a central role in reward
processing (Ikemoto, 2010). One of the major mammalian DA pathways,
known as the mesocorticolimbic DA pathway, connects dopaminergic neu-
rons in the ventral tegmentum to the nucleus accumbens and separately to
the frontal cortex (Carr and Sesack, 2000; Omelchenko and Sesack, 2009;
Van Bockstaele and Pickel, 1995). Two areas in this pathway, the ventral
pallidum and nucleus accumbens, have been found to be major foci for
reward learning and pleasure (Smith et al., 2007). Although the exact func-
tion of DA is still to be determined, we now know that DA plays a much
more complicated role in reward than as simply a pleasure transmitter. Dif-
ferent DA circuits are involved in the different liking, wanting and learning
elements of the reward response, and they interact with a range of other
neurochemical systems. Opioids, serotonin, endocannabinoids and
γ-aminobutyric acid (GABA) also play a role in different components of
reward (Gardner, 2005; Kranz et al., 2010; Van Ree et al., 2000;
Vlachou and Markou, 2010).
Mammalian studies have emphasised the distinction between wanting
and liking of reward. In rodent models, the latter is indicated by distinctive
orofacial expressions of the rodents that are similar to the innate expressions
of humans to food reward and are considered indicative of the affective state
of the animal. Affective neuroscience studies of rodents have indicated
hedonic (liking) reactions, orofacial reactions to food stimuli, to be
192 Eirik Søvik et al.

coordinated by a network of hedonic hotspots distributed within the nucleus
accumbens and ventral pallidum but also in other forebrain and deep
brainstem regions. Microinjections of drugs that activate neuronal opioid,
endocannabinoid or related neurochemical receptors in these hedonic hot-
spots significantly increase the normal number of “liking” reactions to a
sucrose taste (Mahler et al., 2007; Pecin
˜a and Berridge, 2005; Smith and
Berridge, 2005; Smith et al., 2009). Berridge and colleagues blocked DA
systems via antagonists in rats and showed that hedonic reactions to food
reward were as strong as normal but the “wanting” reactions (e.g. eating
more voraciously) had changed. Similarly, stimulation of anatomically sep-
arate regions within the same midbrain areas have been shown to increase
“wanting,” but not affect the rats’ “liking” of the food through their hedonic
responses (Berridge and Valenstein, 1991). These among many other exper-
iments have shown that the liking and wanting psychological components of
reward have distinct neuroanatomical and neurochemical pathways.
Assessment of liking in insects is difficult. For obvious reasons, insects
do not display the orofacial reactions that are considered indicative of liking
in mammals. Consequently, the insect literature tends to discuss not
whether a reward is liked or sought, but simply the valance of a reward
(Galizia, 2014; Knaden et al., 2012; Parnas et al., 2013). Valence describes
stimuli simply by the extent to which they solicit a positive or negative
response from an animal. That response might be approach versus avoid-
ance behaviour, or an appetitive versus a rejection response. The valence
concept is aligned well with the simple bioassays that have dominated much
of the Drosophila literature, and is entirely operational in definition with no
assumption of the internal mental processes of the animal. It may, however,
incorporate both liking and wanting in the observed behavioural responses.
The term valence as applied in the invertebrate literature is very different
from the concept of emotional valence that is sometimes used in human
and comparative psychological literature to describe positive and negative
emotional reactions.
Our objective below is to review what is currently known about the
reward systems of honey bees and fruit flies to assess the degree to which
they might be comparable. This issue is important because honey bees
and fruit flies are the two dominant insect neuroscience models, and it is
important to consider to what degree findings might translate across these
systems despite the significant phylogenetic distance between them. The lit-
erature is currently a little confusing: for a long time, neurobiological sim-
ilarities were assumed across insect orders; however, new findings highlight
193Insect Reward Systems: Comparing Flies and Bees

an increasing divergence in our understanding of honey bee and fruit fly
reward systems.
3. REWARD SYSTEMS IN HONEY BEES AND FLIES
The two major insect models for comparative neuroscience are the
fruit fly and the European honey bee. Since Benzer established Drosophila
as the model organism for behavioural neurogenetics (Weiner, 1999), the
fly has been widely used to study the molecular underpinnings of memory
(Davis, 2005). Early investigations did not aim at understanding the fruit fly
reward system per se, but rather to identify molecular components involved
in memory processing. This research initially focused on aversive condition-
ing (Quinn et al., 1974), but subsequent investigators modified the assays
used to enable the investigation of reward conditioning (Tempel et al.,
1983), which facilitated study of the fruit fly reward system. The honey
bee has arguably been as important a model as Drosophila for studies of insect
learning. Appetitive behaviours in honey bees were first studied by von
Frisch at the turn of the twentieth century (von Frisch, 1914); however,
mechanistic studies of the neurobiology of reward did not begin in earnest
until studies of reward learning using the proboscis extension reflex
(Kuwabara, 1957).
3.1 Experimental approaches
3.1.1 Behavioural paradigms utilised to study reward systems
The most widely used experimental approach to dissect reward systems
in Drosophila is the t-maze. In this apparatus, groups of fruit flies are typically
trained to associate a sucrose reward with an odour stimulus (Tempel et al.,
1983). Afterwards, the valence of the rewarded odour is assessed, by exam-
ining whether the group of fruit flies distributes themselves closer to the
reward-associated odour than a control odour. This is a very different
approach from the most common assay used to assess reward learning with
honey bees: proboscis extension response (PER) conditioning. In PER
experiments, individual honey bees are restrained so that only the antennae
and proboscis are free to move (Bitterman et al., 1983; Kuwabara, 1957).
In this position, honey bees reflexively extend their proboscis if sucrose is
touched to the antenna. During conditioning, honey bees are exposed to
an odour paired with sucrose exposure to the antenna. Post-conditioning,
odour exposure becomes sufficient to elicit proboscis extension, and this
response is considered indicative of reward learning.
194 Eirik Søvik et al.

3.1.2 Fruit fly experiments: Advanced genetic tools allow for careful
investigation of the role of specific neural circuits in reward
The power of Drosophila lies in the advanced genetic tools that have been
developed for this model system. Early fruit fly studies relied on discovering
mutants with interesting defects. This process more or less relied on chance,
as investigators induced random mutations and carefully studied mutants’
behaviour with the hope of discovering phenotypes of interest (this
approach is known as forward genetics). However, this all changed when
the advent of the genomic era allowed Drosophila researchers to investigate
the functions of variants of known genes or gene knockouts in different
experimental paradigms (this is known as reverse genetics). Here, we give
a brief summary of the techniques used in some of the most revealing exper-
iments on reward.
Perhaps the most important single genetic tool has been the incorpora-
tion of the GAL4-UAS driver system in fruit flies (Brand and Perrimon,
1993). This is a powerful method that allows for manipulation of expression
of a gene of interest in selected tissue only. For the GAL4-UAS system to
work, two transgenes are necessary: a GAL4 transcription factor and a
GAL4 upstream activating sequence (UAS). Typically, two constructs
(one containing the promoter region of a gene of interest upstream from
GAL4, and another with the gene of interest downstream from UAS) are
introduced into two separate fly lines. When the gene upstream of GAL4
is transcribed, so is GAL4. The resulting GAL4 protein binds to UAS and
initiates transcription of the transgene and any gene linked to UAS. The
beauty of this system is that GAL4 is only expressed in cells where the
upstream gene is normally expressed. Thus, if GAL4 is downstream of a par-
ticular enzyme or transcription factor, the UAS-linked transgene will only
be expressed in cells naturally expressing the enzyme or transcription factor,
while leaving all other cells unaffected. Since these are two separate trans-
genes, it is possible to maintain separate lines containing GAL4 and UAS
without the transgene being expressed because there is no UAS sequence
for the GAL4 protein to bind to in lines containing GAL4-only and vice
versa. Labs can easily maintain strains without worrying about any ill effects
of the mutants, as the transgenes are ineffectual until both come together in
the same individual. While initially the GAL4-UAS system was used to
ectopically express genes of interest, any RNA of interest can now be
expressed using the system. The tool can be used to express genes from other
species, overexpress genes of interest or even silence expression of specific
genes using RNAi (Dietzl et al., 2007).
195Insect Reward Systems: Comparing Flies and Bees

While the ability to over- and under-express genes has been very fruitful,
perhaps the biggest innovation for neurogenetics research was the use of the
GAL4-UAS system to directly silence and activate selected neurons with
high temporal precision. Thus, a fruit fly can develop and behave normally,
while specific neurons can be activated or silenced in a context of interest.
By expressing a temperature-sensitive variant of the GTPase shibire (shi
ts1
)
under UAS control, it is possible to inhibit neurotransmitter release. At tem-
peratures above 29 °C, shi
ts1
inhibits recycling of synaptic vesicles and causes
a cessation of neurotransmission from neurons expressing GAL4 (Kitamoto,
2001). Because of the temperature-sensitive nature of this allele, GAL4-
expressing neurons function normally at other temperatures, but neuro-
transmitter release can be stopped by increasing the temperature. It is also
possible to directly activate neurons. One of the earliest techniques used
for this purpose was Channelrhodopsin-2 (ChR2) in combination with
GAL4-UAS (Schroll et al., 2006). ChR2 is a light-gated cation-selective
membrane channel (Nagel et al., 2003). When cells expressing the ChR2
protein are illuminated, it causes a rapid depolarisation of any cell where
it is expressed. One drawback of ChR2 is that the cell of interest needs
to be illuminated directly. This works well with translucent larvae, but in
order to use this method in adults, it is necessary to open the pigmented head
capsule to expose neurons in the brain to light, which limits the range of
behavioural experiments a where this technique can be used. It is now also
possible to use a thermogenetic approach similar to shi
ts1
to activate neurons.
Using TrpA1 (a temperature-sensitive cation channel) in the same manner as
ChR2, it is possible to rapidly depolarise neurons by exposing fruit flies to
temperatures above 27 °C(Hamada et al., 2008). Reaching the required
temperature can be achieved very rapidly by heating fruit flies directly with
lasers, allowing for sub-second timing of neuronal activity (Bath et al., 2014).
These techniques have made it possible to examine function and circuitry of
specific neuron populations in fruit flies with unprecedented precision.
3.1.3 Honey bee experiments: Electrophysiology and pharmacology
While very recent advances have been made to make it possible to generate
transgenic honey bees (Ben-Shahar, 2014; Schulte et al., 2014), nothing like
the tools seen in Drosophila are available. Instead, the majority of honey bee
studies have employed behavioural, pharmacological or electrophysiological
techniques. These different approaches allow for very different sets of exper-
iments to be performed. Despite this, until very recently there was broad
agreement about the neurobiology of the reward systems in honey bees
196 Eirik Søvik et al.

and flies, which led to the assumption that reward systems were broadly con-
served across insect orders. Somewhat confusingly, the most recent studies of
neurotransmitter systems in the Drosophila reward system seem to conflict
with classic honey bee electrophysiological and pharmacological studies.
Early electrophysiological analyses by Hammer and Menzel of the neu-
ron VUMmx1 proved to have enormous influence on the comprehension
of the honey bee reward system (Hammer, 1993; Hammer and Menzel,
1995). As we discuss in Section 3.2 below, stimulation of VUMmx1 was able
to substitute sucrose reward in the proboscis extension paradigm. Evidence
suggests VUMmx1 is octopaminergic (Hammer and Menzel, 1998), and
studies of the reward system in honey bees since then have focussed on both
the role of OA and locations where VUMmx1 projects.
Most studies in honey bees, however, have used neuropharmacological
methods to explore circuit functions (Scheiner et al., 2002). The pharmaco-
logical methods applied in honey bees cannot match the resolution and pre-
cision of the neurogenetic methods available for fruit fly research. Many
behavioural studies have applied pharmacological agents to the whole brain
or systemically (Barron et al., 2007b; Farooqui et al., 2004), and even brain
microinjection targets a region rather than a specific circuit. It is also difficult
to completely differentiate between octopaminergic and dopaminergic cir-
cuits with pharmacology. A range of compounds have been identified that
differentially bind to OA or DA receptors, but there remains some overlap
(Beggs et al., 2011; Mustard et al., 2005). This is especially true of the
AmDOP2 and AmOA1 receptors (Beggs et al., 2011). Sadly, no single com-
pound has affinity completely limited to a single biogenic amine receptor
(Mustard et al., 2005); consequently, it is vital in honey bee pharmacological
studies to use a range of agonists and antagonists in behavioural analyses to
tease apart the most likely contributions of different receptor systems to any
given behaviour.
3.2 Neuroanatomy of insect reward systems
Hymenoptera and Diptera diverged in the Carboniferous (more than 300
million years ago) concordant with the divergence of seed plants
(Grimaldi and Engel, 2005). Despite this ancient divergence, many key
aspects of brain morphology and function are apparently conserved between
these two groups, as we describe below. The honey bee brain is an order of
magnitude larger than the Drosophila brain (Fig. 1). The honey bee brain
possesses approximately 1 million neurons compared to approximately
197Insect Reward Systems: Comparing Flies and Bees

100,000 neurons in the fruit fly (Shimada et al., 2005; Witth€oft, 1967). All
the major areas of neuropil recognised in the fruit fly brain are present in the
honey bee brain (Fig. 2). Below, we briefly lay out which neuropils are
thought to be involved in reward processing, in honey bees and flies, and
discuss their supposed functions.
Most studies examining reward in honey bees and fruit flies have done so
in the context of olfactory learning, and therefore, most of our inferences
about the reward system are derived from experiments using an olfactory
stimulus during conditioning. Typically, in an experiment, the animal is
given a reward, usually in the form of sucrose, while being simultaneously
exposed to an odour. While the sucrose is detected by gustatory receptors on
the proboscis or tarsi (Wang et al., 2004), odours are detected via olfactory
receptor neurons (ORNs) in the antenna or the maxillary palps and a signal is
sent to glomeruli in the antennal lobes (ALs). A single ORN projects to one
Central complex
Antennal lobes
Medulla
Mushroom bodies
Lamina
500
m
m
Figure 1 3D models of the honey bee brain and Drosophila brain to scale.
Figures provided by Paulk, and adapted from Paulk et al. (2014) with permission.
198 Eirik Søvik et al.

Figure 2 The reward pathways (olfactory and gustatory circuits) of the honey bee brain
and fruit fly brain. Schematic frontal views of a cross-section of the central honey bee
(A) and fruit fly (B) brain (head capsule and eyes removed). The olfactory (CS) pathway is
depicted in light blue (light grey in the print version). Olfactory neurons send informa-
tion to the brain via the antennal nerve. These neurons form synapses within the glo-
meruli of the antennal lobes (ALs) onto local interneurons (not shown) and projection
neurons conveying olfactory information to the lateral horn (LH) and the mushroom
bodies (MBs). In the honey bee brain (A), the gustatory (US) pathway involves the
VUMmx1 neuron (brown, light grey in the print version), which projects from the gna-
thal ganglion to the LH, AL and MB. VUMmx1 is bilaterally symmetrical, but in this figure
only the right side is shown. Abbreviations: CS, conditioned stimulus; US, unconditioned
stimulus; VUMmx1, ventral unpaired medial neuron 1 of the maxillary neuromere. Figure
adapted from Perry and Barron (2013) with permission.
199Insect Reward Systems: Comparing Flies and Bees

glomerulus only. The signal from multiple ORNs is processed within a glo-
merulus before the output is transmitted to projection neurons (PNs). The
PNs project to the mushroom bodies (MBs) and the lateral horn (LH).
The MBs have long been of interest to researchers because of their
potential role in sensory integration and their location as a junction between
sensory processing centres and premotor centres (Kenyon, 1896). This is also
the structure where honey bees and fruit flies show the most anatomical dif-
ference. Drosophila MBs are rather modest occupying less than 4% of the
brain volume. Each contains approximately 2500 Kenyon cells, with a rather
simple calyx (the region of dendritic input to the MB) composed of two
fused caps that resembles a simple button. By contrast, honey bees MBs
are exuberant, filling more than 20% of the brain volume and each con-
taining more than 170,000 Kenyon cells with complex folded double calyces
(Giurfa, 2013). In the fly, the MB only receives olfactory input, whereas in
honey bees the MB receives olfactory, visual, mechanosensory and gustatory
inputs to different regions of the calyx (Ehmer and Gronenberg, 2002;
Giurfa, 2013; Mobbs, 1982; Strausfeld, 2002). The potential for multimodal
integration in the honey bee MB may be necessary to support the learning of
multimodal and abstract stimulus properties associated with rewards
(Avargue
`s-Weber et al., 2010; Perry and Barron, 2013). The MBs have been
shown to be indispensable for olfactory memory formation in fruit flies
(Heisenberg et al., 1985) and honey bees (Menzel et al., 1974). While there
has been some dispute about the particular function of the MB, Heisenberg
(2003) has argued that outputs from the calyces of the MB are necessary for
olfactory reward learning (Heisenberg, 2003). In mutant flies with mal-
formed MBs, olfactory learning was affected but visual, tactile and motor
learning were not affected (Wolf et al., 1998). Although there is no direct
input from the optical lobes to the MB in flies, a small subset of MB intrinsic
neurons are involved in visual reward learning in flies (Vogt et al., 2014). It is
currently unclear if the MB conveys valence to rewarding stimuli, or if it
functions more generally as a classifier of incoming stimuli.
Recent papers with Drosophila have argued that valence is assigned in the
LH (Galizia, 2014; Parnas et al., 2013). As mentioned above, this neuropil
also receives input from the PNs in the AL. Perhaps the most convincing
evidence for the LH as a valence assignor comes from the discovery of
the gustatory receptor Gr43a in a subset of LH neurons. This receptor
detects fructose in response to changes in haemolymph level in response
to feeding (Miyamoto et al., 2012). Further, the spatial arrangement of neu-
rons projecting to this region from the AL has been shown to be indicative of
200 Eirik Søvik et al.

whether or not they drive approach or avoidance (Knaden et al., 2012).
When multiple competing odours are being detected, the combined valence
is computed in the LH (Parnas et al., 2013). The LH is ideally situated to be a
valence assignor, as it is a premotor area and therefore activation of neurons
in this area could activate motor systems for either approach or avoidance.
Another potentially important neuropil for reward processing is the gna-
thal ganglion (GNG). This ganglion receives input from chemosensory
receptor neurons on the proboscis and tarsi (Stocker, 1994). Large widely
branching octopaminergic neurons with their soma in the GNG appear
to be a conserved feature of insect brain anatomy across diverse insect orders.
In honey bees, Drosophila,Locusta migratoria and Periplaneta americana, the
GNG contains a very small number of octopaminergic neurons (Bra
¨unig,
1991; Bra
¨unig and Burrows, 2004; Sinakevitch and Strausfeld, 2006;
Sinakevitch et al., 2005). These neurons, 26 in Drosophila (Sinakevitch
and Strausfeld, 2006) and 12 in the honey bee (Sinakevitch et al., 2005),
are located in small and sparsely distributed clusters (Bra
¨unig, 1991;
Bra
¨unig and Burrows, 2004; Sinakevitch and Strausfeld, 2006;
Sinakevitch et al., 2005). Many of these neurons have very extensive inner-
vations so that most, if not all, regions of neuropil have octopaminergic input
(Busch and Tanimoto, 2010; Busch et al., 2009; Sinakevitch and Strausfeld,
2006; Sinakevitch et al., 2005).
In what has become a landmark study, Hammer (1993) recorded from
VUMmx1 while honey bees were trained in the restrained proboscis exten-
sion paradigm (Bitterman et al., 1983; Kuwabara, 1957) to learn the associ-
ation between a novel odour and sucrose reward. VUMmx1 responded
strongly to sucrose with prolonged firing (Hammer, 1993). This finding
was replicated by Schr€oter et al. (2007), although the duration of the
response to sucrose was less. In a PER paradigm (Bitterman et al., 1983),
honey bees were conditioned such that a novel odour immediately preceded
sucrose presentation, VUMmx1 then responded to the conditioned odour as
well as sucrose reward (Hammer, 1993). In this paradigm, the muscle con-
trolling proboscis extension (M17) could be conditioned to respond to a
novel odour that was forward-paired with sucrose delivery to the antennae
and mouthparts after just a single learning trial (Hammer, 1993). The M17
muscle could also be conditioned if the sucrose unconditioned stimulus dur-
ing training was replaced by a transient supra-threshold depolarization of
VUMmx1 (Hammer, 1993). VUMmx1 did not directly activate muscle
M17 or proboscis extension, but the neuron responded to both primary
sucrose rewards and odour stimuli associated with reward, and the neuron
201Insect Reward Systems: Comparing Flies and Bees

was part of the mechanism by which proboscis extension could be condi-
tioned to respond to stimuli predictive of reward. Menzel (2001) drew a par-
allel between these properties of VUMmx1 and the properties of the
dopaminergic neurons in the mammalian midbrain which also respond to
primary and conditioned rewards, and are organisational of reward-seeking
behaviour (Schultz, 1997, 2002, 2007).
In Drosophila, a single large neuron OA-VUMa2 that shows octopamine-
like immunoreactivity and expresses tyramine decarboxylase (the enzyme
converting tyramine to octopamine) has a large cell body in the GNG
and projects to AL, MB calyces and LH (Busch et al., 2009). This is signif-
icant because the morphology of this neuron is strikingly similar to that of a
VUMmx1 (Hammer, 1993; Hammer and Menzel, 1995). However, as of
yet, it is not clear if any of the OA-VUM neurons in flies perform the same
function as VUMmx1 in bees (Burke et al., 2012).
While it is often assumed that memories are stored and formed in areas
where input from disparate regions converges (e.g. after several steps of
information processing), information does not flow in one direction in
the Drosophila or honey bee nervous systems. In fact, neurons from the
MB also give feedback to target PNs in the AL (Hu et al., 2010). This means
that there is the potential for reward learning to occur within the AL. So far,
it has been demonstrated in flies that plasticity in odour coding in the AL is
sufficient to result in change of behaviour independent of activity in the MB
(Thum et al., 2007). However, only appetitive memory storage within the
AL has been demonstrated, while aversive conditioning failed to produce a
similar result (Thum et al., 2007). Based on these findings, it is important that
we remain cognisant of the possible involvement of feedback to areas of sen-
sory input if we want to understand how insect reward systems work.
3.2.1 Neuroanatomy of larval reward pathways
So far, we have only considered the structure of the nervous systems of adult
insects, but all holometabolous insects also have a larval form with its own
structurally distinct nervous system. The adult and larval forms represent two
distinct organisms with very different lifestyles and ecologies (Truman and
Riddiford, 1999), and their nervous systems reflect this reality. There are
some structural overlaps between the two nervous systems; both larvae
and adult fruit flies have MBs, for example, but during metamorphosis
the majority of MB-projecting neurons are pruned away and new ones
are formed (Technau and Heisenberg, 1982). This is necessary as the sensory
202 Eirik Søvik et al.

input neurons for the adult form are not the same as for the larvae (Chapman,
2013). This dramatic neural transformation may contribute to the memory
discontinuity that seems to exist between larval and adult insects. It is com-
monly reported that appetitive memories formed in larvae are abolished dur-
ing metamorphosis (Barron and Corbet, 1999). Therefore, when discussing
Drosophila reward pathways we are potentially talking about at least two dis-
tinct reward systems: the larva and adult.
While honey bees have free-living larvae, they are not “free living” in
the same way one would describe Drosophila larvae. Rather than foraging
and fending for themselves like Drosophila larvae, honey bee larvae are very
fragile and remain mostly immobile in a wax cell where they are fed by adult
bees. Currently, we know nothing about the potential for honey bee larvae
to associate rewards with stimuli, or how they might otherwise react to
rewards.
3.3 Octopamine signals reward in insects
3.3.1 Octopamine signals reward in honey bees
Early studies in honey bees implicated OA in reward processing. Application
of OA to the brain increased bees’ sensitivity to sugars (Mercer and Menzel,
1982) and enhanced storage acquisition and retrieval of reward associations
(Erber et al., 1993; Menzel et al., 1999). These studies directed attention to
what possible octopaminergic neurons might be involved. Shortly thereaf-
ter, an immunohistochemical study provided some evidence that the neuron
identified by Hammer (1993), VUMmx1, was octopaminergic (Kreissl et al.,
1994). Concluding that VUMmx1 was octopaminergic, Hammer and
Menzel (1998) investigated the consequences of OA microinjection into
the major zones of projection of VUMmx1: MB, AL and LH. Both the
MB and AL had already been shown to be important foci for learning
and memory (Menzel et al., 1974). Microinjection of OA into either the
AL or MB, but not the LH, paired with presentation of an odour was suf-
ficient to condition proboscis extension to the odour. This suggested that
microinjection of OA could substitute for sucrose in reward learning
(Hammer and Menzel, 1998). Repeated presentation of odour forward-
paired with microinjection of OA into the AL resulted in a smooth progres-
sive acquisition of a learned PER to the odour (Hammer and Menzel, 1998).
In the MB calyx, however, repeated training with odour forward-paired
with microinjection of OA resulted in a learned PER only 20 min post-
training as if a consequence of a consolidation process rather than an
203Insect Reward Systems: Comparing Flies and Bees

acquisition process. OA microinjection into the LH paired with odour did
not result in any learned proboscis extension to the odour.
Hammer and Menzel’s (1998) study proved to be extremely influential;
it was taken as strong evidence that VUMmx1 is octopaminergic. It also
positioned octopaminergic circuits within the MB and AL as key elements
of the honey bee reward system. A model of insect learning developed in
which simultaneous presentation of a conditioned odour stimulus with an
unconditioned rewarding stimulus such as sugar will activate both the olfac-
tory stimulus pathway and reward pathways in the brain (Menzel, 2001). For
the olfactory stimulus, this will include a specific pattern of neural activity in
the glomeruli of the AL, PNs from AL, and MB for the CS (Fig. 3). For the
US, this will include motor pathways controlling proboscis extension, and
also VUMmx1. VUMmx1 releases OA into the AL and MB calyces where
the neuromodulator acts to change the likelihood that future odour presen-
tation will be sufficient to activate proboscis extension. This model focused
attention on the role of OA in the MB and AL for reward learning, and
Projection
neurons
Antennal lobe
Antenna
Olfactory receptor
neurons
Odour
CS
Glomeruli
Mushroom body extrinsic neurons
Mushroom body
Lateral horn
Approach
Avoidance
Kenyon cells
Figure 3 Circuit model of odour memory. Odour stimuli activate different sets of glo-
meruli within the antennal lobe. Odour information is conveyed to the MB by projection
neurons that synapse with the MB calyx and separately within the lateral horn. Odours
are represented in the MBs as distinct patterns of activity across sets of Kenyon cells.
Extrinsic MB output neurons are connected to the Kenyon cells by latent synapses.
Figure adapted from Perry and Barron (2013) with permission.
204 Eirik Søvik et al.

perhaps unfortunately it led to a perception that the LH was not involved in
reward learning. Models of the insect reward system, and the possible roles of
the LH and VUMmx1 within it, have continued to evolve as we
discuss below.
In the AL, modulatory actions of OA released by VUMmx1 very likely
contribute to the changes in odour coding across glomeruli that occur in this
region when odours are paired with sucrose reward (Denker et al., 2010;
Faber et al., 1999; Locatelli et al., 2013; Rath et al., 2011). Differential con-
ditioning with one rewarded and one unrewarded odour causes the patterns
of glomerular activation for the two odours to diverge, making the patterns
for these odours more distinct after training (Faber et al., 1999; Fernandez
et al., 2009; Galizia and Menzel, 2001; Locatelli et al., 2013). Additionally,
overall activity for the reward-associated odour increases in both the ALs
(Denker et al., 2010; Faber et al., 1999) resulting presumably in increased
discriminability, changing the valence of the odours and increasing their
salience (Fernandez et al., 2009). OA modulates network activity within
the AL (Rein et al., 2013), and OA signalling via the OA receptor AmOAR
in the AL is necessary for the acquisition and recall of odour–sucrose asso-
ciations (Farooqui et al., 2003).
For odour learning, the cross-glomeruli pattern of activity representing a
specific odour activates a specific pattern of PNs, which project to the LH
and MB. In the MB, PNs synapse with Kenyon cells within the lip of the
calyx. The primary role of the MB is odour identification (Galizia, 2014).
Each identifiable odour activates a distinct pattern of Kenyon cells, with
each odour sparsely coded as a cross fibre pattern across the very large num-
ber of Kenyon cells. Like the AL, odour coding within the MB is also plastic
and serves to increase the valence and discriminability of rewarded odours
(Szyszka et al., 2005, 2008). Odour pairing with sucrose results in a
stabilisation and strengthening of the activity pattern within the Kenyon
cells, whereas non-rewarded odour presentations lead to a weakened odour
response in the Kenyon cell population (Szyszka et al., 2008). Kenyon cells
output to extrinsic neurons which project to the LH, and within this con-
nection matrix there is an additional layer of learning-related plasticity
(Strube-Bloss et al., 2011). Reward learning increases the number of extrin-
sic neurons responding to an odour and causes the activity patterns arising
from rewarded and non-rewarded odours to diverge (Strube-Bloss et al.,
2011). One identified neuron, PE1, which is inhibitory of the downstream
LH, showed reduced firing to odours that have been paired with sugar
205Insect Reward Systems: Comparing Flies and Bees

reward, but not to unrewarded odours (Menzel and Manz, 2005; Okada
et al., 2007).
There is abundant evidence showing OA signalling in the MB is
involved in reward learning in the bee (Hammer and Menzel, 1998), but
here the precise role of VUMmx1 is not clear. Heisenberg (2003) argued
convincingly that (in Drosophila at least) learning required cAMP signalling
within Kenyon cells and that the necessary locus of synaptic activity for
reward learning was the outputs of the Kenyon cells in the lobes of the
MB where they synapse with the extrinsic neurons (Fig. 3). VUMmx1,
however, arbourizes within the calyx of the MB rather than the lobes,
and within the calyx arbours are rather sparse and may be parasynaptic with
Kenyon cell fibres rather than making one-on-one connections (Sinakevitch
et al., 2005). OA released by VUMmx1 could generate cAMP signals within
Kenyon cells in the region of the calyx, but it seemed unfeasible for a cAMP
signal to diffuse from there to the entire length of the axon of the Kenyon
cells to affect pre-synapses at the axon terminals in the lobes (Heisenberg,
2003). Even so, OA released by VUMmx1 within the calyx could be imag-
ined to influence the activity of Kenyon cells by altering weights of synaptic
connection within the calyx. The calyx is a region of synaptic contact
between the Kenyon cells and the GABA-immunoreactive inhibitory neu-
rons of the protocerebral tract which provides inhibitory feedback from the
MB lobes to the calyces (Bicker et al., 1985; Gr€unewald, 1999a,b). Further,
in Drosophila Kenyon cells in the calyx are not exclusively post-synaptic
(Christiansen et al., 2011), and Kenyon cell dendrites contain both pre-
and post-synaptic terminals. This raises the possibility of feedback connec-
tions between Kenyon cells or with PNs. OA released into the calyx could
modulate synaptic connections there to alter the strengths of feedback signals
and thereby alter activity within the Kenyon cells (Strube-Bloss et al., 2011).
3.3.2 Octopamine in fruit flies
After the experiment demonstrating that the octopaminergic VUMmx1
neuron could replace positive appetitive stimuli in a learning experiment
(Hammer, 1993), it was widely assumed that reward learning involved
octopaminergic signalling in insects (Barron et al., 2010; Hammer and
Menzel, 1998; Mizunami et al., 2009; Perry and Barron, 2013;
Schwaerzel et al., 2003; Unoki et al., 2005, 2006). This was surprising
because in mammals it was assumed that the main neurochemical signal
for reward is DA (Schultz, 2002; Schultz et al., 1997). Early studies with Dro-
sophila supported the emerging hypothesis that in insects, reward was
206 Eirik Søvik et al.

signalled by OA (Honjo and Furukubo-Tokunaga, 2009; Schwaerzel et al.,
2003). Schwaerzel et al. (2003) found that mutants lacking tyrosine-beta-
hydroxylase (a synthesis enzyme necessary for production of octopamine)
were unable to form appetitive memories of sucrose paired with an odour,
indicating that OA was necessary for the formation of appetitive memories.
Further, the behavioural phenotype of these mutants could be rescued by
either feeding fruit flies OA or by expressing tyramine-beta-hydroxylase
thermogenetically. Perception of sucrose was not affected in either of these
mutants, suggesting that the role of OA is not directly related to sensing
sucrose. Schroll et al. (2006) did a series of experiments in Drosophila larvae
using ChR2 in combination with tyramine-beta-hydroxyase under GAL4
control to show that activating octopaminergic neurons alone is sufficient
to generate positive valence for an odour stimulus. Using the shi
ts1
method,
Honjo and Furukubo-Tokunaga (2009) found that when OA release was
inhibited in larvae, formation of appetitive memories was blocked. Taken
together, these findings added strong support to the hypothesis that in insects
OA neurons signal reward.
However, unlike in bees, the origin of the key octopaminergic signal has
remained elusive. Several octopaminergic neurons project to the MB and
LH as in honey bees (Busch et al., 2009; Sinakevitch and Strausfeld,
2006), and the MB in particular contains many octopaminergic receptors
(Han et al., 1998). Interestingly, activating individual GNG OA neurons
that innervate the MB tightly temporally paired with an odour (including
OA-VUMa2) did not establish appetitive memories (Burke et al., 2012).
Blocking these neurons during training did not inhibit memory formation
either (Burke et al., 2012). It would seem that the ventral paired and
unpaired neurons in Drosophila have a very different function than
VUMmx1. The research of Schwaerzel et al. (2003) suggests the Drosophila
reward system involves OA signalling to the MB, but the specific cell
populations generating that signal are currently unknown. Individual
VUM neurons are not sufficient in fruit flies for reward learning. Currently,
we cannot say whether this means they are not involved in reward signalling,
or whether the fruit fly octopaminergic system fails to function correctly
when dissected to individual components.
New research with Drosophila suggests that the evaluation of odour
valence occurs within the LH (Galizia, 2014). Galizia (2014) proposes the
MB can be considered loci for odour identification whereas the LH is the
locus for determining whether the odour is rewarding or punishing, and
whether the animal is in a motivational state to respond (Galizia, 2014). This
207Insect Reward Systems: Comparing Flies and Bees

new perspective places the LH as key to a functional reward system. If this
view is correct, it presents a difficulty in reconciling Hammer and Menzel’s
(1998) classic studies with microinjection of OA into honey bee brain neu-
ropils. Given the well-established roles for OA in arousal, food seeking and
food learning in bees, why did microinjection of OA into the LH not estab-
lish reward learning? The answer may be that the principal known input to
the LH from the MB is inhibitory (Okada et al., 2007; Rybak and Menzel,
1998). The inhibitory output from the MB in response to an odour decreases
after that odour has been paired with reward (Okada et al., 2007; Rybak and
Menzel, 1998). This would release the LH from inhibition and enable motor
pathways to be activated. In bees, depolarisation of VUMmx1 could poten-
tially release OA at three points through the honey bee reward system caus-
ing coordinated changes in the circuit to alter the valence of a stimulus.
A microinjection of OA to the LH alone, however, would not release
the LH from inhibition by the MB and would not be sufficient for a learned
PER to a stimulus to be expressed.
3.4 Dopamine in insect reward
3.4.1 Dopamine signals punishment in insects
The focus on OA as a neuromodulator of reward learning marked a devi-
ation from mammalian research in which DA was clearly the principal neu-
rochemical system of reward signalling, and OA is only present in trace
amounts. However, the distinction between insect and mammals became
even sharper after early fruit fly studies started finding that DA signalled pun-
ishment instead of reward. DA was first implicated in aversive learning when
Tempel et al. (1984) demonstrated that mutants lacking dopa-decarboxylase
(synthesis enzyme for dopamine) were impaired in an aversive learning par-
adigm utilising electric shocks. This finding was corroborated by experi-
ments showing that aversive learning was impaired when dopaminergic
neurons were silenced with shi
ts1
(Schwaerzel et al., 2003). Inhibiting DA
release with shi
ts1
impaired formation of aversive memories in larvae as well
(Honjo and Furukubo-Tokunaga, 2009). Further, activating dopaminergic
neurons with ChR2 in larvae was sufficient to replace aversive stimuli
(Schroll et al., 2006). Since these experiments, several studies have narrowed
down the anatomical sites necessary for aversive memory formation (Aso
et al., 2010, 2012; Claridge-Chang et al., 2009). Based on these studies, it
became clear that in fruit flies, specific DA neuron populations signal
punishment.
208 Eirik Søvik et al.

Pharmacological studies in honey bees have shown that DA antagonists
inhibit aversive learning in honey bees (Vergoz et al., 2007; Wright et al.,
2010). This was first shown with electric shock (Vergoz et al., 2007) and later
with distasteful food (Wright et al., 2010). In these experiments, treatment
with antagonists of OA receptors impaired reward learning but left punish-
ment learning intact, whereas treatment with DA receptor antagonists
impaired punishment learning but left reward learning intact. Consistent
with this story, pharmacological studies in crickets (Gryllus bimaculatus)
(Mizunami et al., 2009; Unoki et al., 2005, 2006) suggest that OA signals
reward, while DA signals punishment (but see Agarwal et al., 2011). How-
ever, as we discussed in Section 3.1.3, we caution that the pharmacological
agents applied are not entirely specific to either OA or DA receptors (Beggs
et al., 2011). Careful studies applied multiple OA receptor and DA receptor
antagonists in order to differentiate the function of the two systems (Vergoz
et al., 2007), but even this refined approach would not entirely separate the
functions of different individual neural circuits or receptors. This has only
become possible thanks to new neurogenetic tools in Drosophila that have
allowed exploration of the functions of specific dopaminergic signals. These
new studies have shown that some dopaminergic neurons are involved in
aversive learning, and others in reward learning.
3.4.2 Dopamine signals some aspects of reward
New evidence from fruit flies has shown that DA is involved in some aspects
of reward learning. Two mutant genotypes defective in DopR (a D1-like
DA receptor, also known as dDA1) in the MB and central complex dem-
onstrated impaired abilities to form appetitive memories (Kim et al.,
2007). This defect could be rescued by ectopic expression of DopR in
the MB. The same mutants displayed similar behavioural defects in larvae
(Selcho et al., 2009). Based on these results, it was clear that dopaminergic
signals in the MB via the DopR receptor played a role in appetitive learning.
In Drosophila, it now seems that DA signals nutritive value as reward
while OA signals sweetness as reward. Feeding fruit flies sucrose transiently
activates neurons in the protocerebral anterior medial (PAM) cluster within
the inferior neuropils (Liu et al., 2012). This is a bundle of approximately
100 dopaminergic neurons that primarily project to the medial lobes of
the MB (Liu et al., 2012). Activating these neurons with TrpA1 could
replace sucrose during appetitive conditioning (Liu et al., 2012), suggesting
that these neurons are involved in signalling some aspect of food reward.
Further, conditioning with TrpA1 activation of PAM was not impaired
209Insect Reward Systems: Comparing Flies and Bees

in fruit flies lacking tyramine-beta-hydroxylase (and therefore lacking OA),
suggesting that these dopaminergic neurons are functionally downstream
from octopaminergic reward signalling. Burke et al. (2012) further dissected
the role of OA and DA in reward with a series of ingenious experiments with
nutritive and non-nutritive sugars; both sweetness and caloric content are
sufficient to induce appetitive memories in fruit flies (Burke and Waddell,
2011; Fujita and Tanimura, 2011). By training fruit flies with OA neurons
silenced with shi
ts1
using arabinose (which is only sweet) and sucrose (which
is both sweet and nutritious), they found that the role of OA was to signal
sweetness, not nutritive content. Based on this, they concluded that the role
of OA in the fruit fly reward system is to signal sweetness, while caloric value
is signalled by other means. The neurons in the PAM cluster respond to OA
via the OA receptor OAMB (α-adrenergic-like receptor) and are thus func-
tionally downstream from the OA signal of sweetness (Burke et al., 2012).
Further, octopaminergic input is necessary to inhibit aversive DA signals.
A dopaminergic neuron in the protocerebral posterior lateral (PPL1) cluster
known as MB-MP1 (mushroom body pedunculus-medial lobe arbourizing
neuron 1) gives aversive input into the medial lobe of the MB (Aso et al., 2010).
OA-dependent reward signalling depends on OCTβR2 (β-adrenergic-like
OA receptor) to inhibit the activity of MB-MP1 (Burke et al., 2012). This
demonstrates that octopaminergic input is necessary to two separate sets of
dopaminergic neurons (PAM and PPL1) in order for appetitive conditioning
to take place. In fruit flies, a picture has now emerged of DA as the principle
signal of nutritive reward, and OA circuits activated by sweet taste and oper-
ating upstream of DA signals. The emphasis on DA in reward in fruit flies
stands in contrast to the long-standing emphasis on OA in reward in bees.
While the neurogenetic studies in Drosophila have identified specific DA sig-
nals for reward in insects, so far nothing is really known about the role of
DA in reward signalling in honey bees. While a recent study has shown that
there is at least the capacity for interaction between OA and DA systems in
the honey bee MB (McQuillan et al., 2012), there is as of yet no evidence to
show that it plays a role in reward signalling. This marks a presently unre-
solved divergence in our understanding of the reward systems of honey bees
and flies.
3.4.3 Dopamine signals presence of amino acids
All experiments discussed thus far have used carbohydrate rewards for sweet-
ness. It is, therefore, not clear how these findings might translate to other
food items, such as proteins. Adult Drosophila carefully regulates both their
210 Eirik Søvik et al.

protein and carbohydrate intake (Lee et al., 2008), so it would be strange if
only carbohydrates, but not protein, were rewarding to these animals. Initial
experiments have found that, at least in larvae, amino acid intake is regulated
by three dopaminergic neurons (Bjordal et al., 2014). When Drosophila lar-
vae were presented with a diet deficient in lysine and tryptophan, it caused
the activation of three dopaminergic neurons that are both necessary and
sufficient for the larvae to reject their current food source and start looking
for a new food source, which could be considered reward-seeking behav-
iour (Bjordal et al., 2014). It remains to be seen if this is generalizable to
adults, and if these neurons feed into the same pathways as carbohydrate
rewards. Regardless, these findings are highly suggestive of DA as a signal
for nutritive value in fruit flies regardless of nutrient group.
3.5 Modulation of reward seeking
In Drosophila, two homologues of the mammalian neuropeptide Y are tran-
scribed in response to hunger and affect fruit fly behaviour. One of the func-
tions of these neuropeptides is to motivate fruit flies to seek out appetitive
stimuli when they are hungry. The first of these two homologues is neuro-
peptide F (dNPF), which is primarily expressed in the brain and midgut of
larva and adult alike (Brown et al., 1999). Satiated fruit flies respond less to
food-associated odours than hungry flies. This is due to tonic signalling from
inhibitory dopaminergic neuron MB-MP1. However, when fruit flies are
hungry, dNPF is expressed, and this signal prevents tonic DA release from
MB-MP1 thus making the fruit fly more responsive to food-associated cues
(Krashes et al., 2009). Thus, NPF regulates a dopaminergic circuit that pre-
vents satiated fruit flies from seeking out food. Further, dNPF neurons are
activated by sex pheromones (Gendron et al., 2014) and modulated neural
and behavioural responses of male flies to copulation as a reward (Shohat-
Ophir et al., 2012) indicating that in the context of sexual reward NPF mod-
ulates reward responses also. The second homologue of neuropeptide Y, the
short neuropeptide F (sNPF), has been shown to be necessary for formation
and recall of appetitive memory (Knapek et al., 2013). Knocking down
sNPF in the MB Kenyon cells or in the neurons of α/β- and γ-lobes with
RNAi caused significant impairment of the ability to form appetitive mem-
ories. However, knocking down the sNPF receptor in Kenyon cells did not
affect olfactory memory, suggesting that the target is elsewhere. Sucrose
preference was not affected in these mutants. sNPF also affects neuronal
response directly in the AL (Root et al., 2011). Thus, it appears in fruit flies
211Insect Reward Systems: Comparing Flies and Bees

that homologues of neuropeptide Y function as a context signal of hunger
state, rather than a direct signal of reward. It can, therefore, modulate the
valence assigned to a food reward by a fruit fly given the present context
the fruit fly operates in (Beshel and Zhong, 2013).
Honey bees have homologues of both dNPF and sNPF that are
expressed in relation to food. However, they have not yet been shown to
mediate food intake (Ament et al., 2011). At this stage, it would be reason-
able to assume that they perform similar functions in the honey bee as in
the flies.
3.6 Non-food rewards
While non-food rewards have received a great deal of attention in mamma-
lian studies (O’Connell and Hofmann, 2011), very little is known of how
non-food rewards are signalled in the insect brain. This may be an important
oversight. When fruit flies are trained to an odour that signals the cessation of
an electric shock, they attribute a positive valence to this odour (Tanimoto
et al., 2004). Interestingly, neither dopaminergic nor octopaminergic neu-
rons affect this form of learning (Yarali and Gerber, 2010). This suggests that
olfactory relief learning probably occurs via another, as of yet, undiscovered
pathway. A recent study demonstrated that OA might be involved in aver-
sive behaviours in a modified conditioned place preference paradigm
(Agarwal et al., 2011). In their experiment, honey bees did less well at learn-
ing to avoid an area paired with electric shock when treated with OA antag-
onists. While not directly examining reward, it suggests that the view that
OA only signals rewards in insects may not be correct. In flies, responses
to sexual reward are affected by NPF (Gendron et al., 2014; Shohat-
Ophir et al., 2012), which suggests that examination of different forms of
reward may reveal new modulatory pathways altering reward responses.
3.7 Summary of findings from honey bees and fruit flies
The reward systems of honey bees and fruit flies have both been very well
studied. They are similar, but not the same. Evidence gathered to date sug-
gests DA appears to be important for reward signalling in flies, but no evi-
dence has yet linked DA to reward signalling in bees. Considering the
conserved nature of dopaminergic reward signalling (Barron et al., 2010),
is this a genuine biological difference? The findings in the fruit fly have been
generated using fine-grained neurogenetic tools designed to alter the func-
tion of very specific circuits. The pharmacological tools employed in honey
212 Eirik Søvik et al.

bees thus far to probe the OA and DA systems lack the anatomical and neu-
rochemical specificity to adequately distinguish between the two (Beggs
et al., 2011; Mustard et al., 2005). It is, therefore, possible that specific
DA circuits could also play important roles in reward in the bee. As a second
point of confusion, while the GNG neuron VUMmx1 appears to be suffi-
cient for reward learning in bees, comparable (and likely homologous) neu-
rons in the fruit fly do not appear to play an important role in learning of
food reward (Burke et al., 2012). Potentially, this could reveal a significant
difference in the circuitry of honey bee and fruit fly reward systems and
poses some urgent unanswered questions. The structural similarity of the Dro-
sophila octopaminergic OA-VUMa2 neuron with the honey bee
VUMmx1 is so striking it seems perplexing that the two neurons do
not have similar functions. We will, however, urge caution in the inter-
pretation of the significance of this difference until the responses have been
studied with similar behavioural assays. Currently, the VUMmx1 honey
bee study measured successful learning based on the activation of the pro-
boscis extension muscle (M17), while in the fruit flies this was measured as
an approach by the whole animal. It could be that the octopaminergic
GNG neurons are only sufficient to elicit proboscis extension as a response
in reward learning, while additional reward signalling neurons are neces-
sary for approach conditioning.
4. REWARD SYSTEMS AND SOCIAL EVOLUTION
The reward system is a major organiser and motivator of behaviour,
and it is, therefore, not surprising that the reward system has been implicated
in many different forms of social behaviour in both mammals and insects.
Examples of social behaviours that are linked to reward processing include
parental care, juvenile social play, adolescent social interactions, sexual activ-
ities, affiliative behaviours and aggressive territorial behaviours (O’Connell
and Hofmann, 2011). Neurobiological analyses suggest that new forms of
social behaviour can evolve by changes in the nature of stimuli or actions
that activate a pre-existing reward system. This could either involve evolu-
tionary changes such that the reward system becomes activated by social
stimuli to promote new forms of social affiliation, or alternatively that exis-
ting reward-directed behaviour is modified to enable new forms of social
behaviour. In mammals, there is evidence of the former, whereas in social
insects currently there is more evidence for the latter.
213Insect Reward Systems: Comparing Flies and Bees

For mammals, the most pertinent example comes from the neurobiolog-
ical analyses of social pair bonding in voles (genus Microtus). Prairie voles
Microtus ochrogaster are socially monogamous and form enduring pair bonds
between males and females, whereas the closely related meadow and mon-
tane voles Microtus pennsylvanicus and Microtus montanus form weak pair
bonds and consequently are more solitary and promiscuous (Williams
et al., 1992). Differences in the strength of the pair bond have been related
to differences between the species in the density and distribution of vaso-
pressin (males) or oxytocin (female) receptors in specific brain regions. In
prairie voles, males express vasopressin receptors at high densities in regions
of the brain known to be involved in hedonic reactions and pleasure
(Donaldson and Young, 2008). Similarly, females express oxytocin recep-
tors at high density in the nucleus accumbens. Vasopressin and oxytocin
are involved in social recognition in mammals (Ferguson et al., 2000,
2001; Young and Wang, 2004) and are released in response to the specific
odour signatures of familiar individuals. Young and colleagues argue that the
neuroanatomical basis of the strong pair bond in prairie voles is the strong
anatomical coupling of the mesolimbic dopaminergic reward system and the
peptidergic social recognition systems (Young and Wang, 2004; Young
et al., 2005). Sex with a known partner would activate both systems,
resulting in the formation of a conditioned preference for the partner. In
the non-monogamous species, the reward system and social recognition sys-
tems are both active, but they are only weakly coupled; the inference being
that the specific social recognition cues of a sexual partner are less rewarding
in these species (Donaldson and Young, 2008; Young and Wang, 2004;
Young et al., 2005).
This classic mammalian example highlights how new forms of social
interaction can evolve by the linking of social recognition cues to the reward
system to alter the valence of recognition cues and thereby promote social
affiliation. There is currently no evidence that a similar mechanism has con-
tributed to the evolution of the insect societies, but this may be because thus
far we have not looked. Nestmate recognition systems are well developed
across social insects (Breed, 1983; Breed et al., 1984, 1988, 1994; Page
and Breed, 1987), and in one species of paper wasp (Polistes fuscatus) individ-
ual recognition has been demonstrated based on individually distinctive
facial markings (Tibbetts, 2002). In many species of social insect, nestmates
instinctively cluster and readily exchange food or groom each other demon-
strating a very high level of innate social affiliation and cooperation (Seeley,
1989, 1995), but currently it is not clear if the insect reward system is
214 Eirik Søvik et al.

activated by social affiliation or whether social recognition cues interact with
the reward system for any insect species.
In honey bees, there is evidence that new forms of social behaviour have
evolved by modification of existing reward-directed behaviour. Specialised
behaviour supporting social foraging is regulated by octopaminergic systems
that feature in solitary foraging roles in other insects. As we have discussed
above in both honey bees and flies, OA circuits mediate the perception of
sucrose reward, learning of sucrose and hunger (Hammer, 1993; Long and
Murdock, 1983; Perisse et al., 2013; Scheiner et al., 2006, 2014). In honey
bees, OA also modulates social foraging. OA treatment decreases the mean
age at which honey bees begin foraging (Barron et al., 2002; Schulz et al.,
2002) and changes how honey bees react to social stimuli within the hive to
promote foraging (Barron and Robinson, 2005; Barron et al., 2002).
Returning foragers sometimes dance as a social signal to communicate to
nestmates the location and value of resources needed by the hive. The dance
is heralded as the only known example of symbolic communication in
insects and represents a remarkable new form of social communication that
only occurs in the genus Apis. Dance behaviour of returning nectar and pol-
len foragers was modulated by OA in a manner that was dose dependent and
could be blocked by the OA receptor antagonist mianserin (Barron et al.,
2007a). Dance parameters reporting the value of located food sources (dance
circuit vigour and dance circuit number) were by far the most sensitive to
OA treatment, while positional information represented in dances was
largely unchanged, suggesting that OA altered the communication of food
value in dances rather specifically (Barron et al., 2007a). Similar effects were
seen following cocaine treatment (Barron et al., 2009), which alters biogenic
amine reuptake in the insect brain (Borue et al., 2010; Søvik, 2013; Søvik
and Barron, 2013; Vickrey et al., 2009). Since the dance signals a workers’
assessment of reward quality, this may be the closest measure we have in
insects to assess an affective state: the subjective assessment of the quality
of a reward.
When honey bees forage for the colony, they do not consume the
resources they have gathered; rather, they immediately surrender them to
their nest mates in the hive or deposit the floral resources in cells in the col-
ony. This is especially true for pollen foragers. Forager honey bees do not
consume pollen, and the gathered pollen is collected in corbiculae on the
hind legs before being deposited in storage cells in the colony (Seeley,
1995; Winston, 1987). Pollen foragers even leave the colony satiated with
a crop full of honey from the hive to fuel their pollen-collecting trip (A.B.
215Insect Reward Systems: Comparing Flies and Bees

Barron, unpublished). Dance is also purely of social benefit: the signal serves
to recruit nestmates to any profitable resource discovered by a forager. Both
of these forms of social behaviour appear linked to the honey bee reward
system.
How these new forms of social foraging behaviour may have evolved
from pre-existing foraging behaviour of the imagined solitary or sub-social
ancestor of the honey bee is a matter for speculation. In his considerations of
animal motivation and drive, Tinbergen (1951) imagined that reward-
seeking behaviour was generated by internal motivational energy that built
up in an animal over time and directed behaviour towards the attainment of
specific resources or circumstances. This internal energy was only released
by the execution of a specific consummatory act that marked the successful
attainment of the sought-for outcome (Tinbergen, 1951). One interpreta-
tion of the evolution of social foraging could be that the pre-existing
reward-seeking system has been exapted by the evolutionary process to
respond to the collection of food as a consummatory act more than the con-
sumption of food. Tinbergen’s terminology has fallen out of usage as neu-
robiological analyses of reward seeking have developed; however, it has
become clear from current comparative neurobiological analyses that differ-
ent elements of the reward system are activated by different stimuli, which to
use Tinbergen’s terminology could be related to different consummatory
acts. As we have discussed, new findings from Drosophila have shown that
sweet taste and nutrition activate different elements of the reward system:
sweet taste is signalled by specific (but presently unidentified) OA circuits,
and the nutritive value of reward is signalled by identified dopaminergic cir-
cuits (Burke et al., 2012). We can speculate that increasing the valence of
sweet taste as a reward might yield an animal that is more motivated to gather
food than to consume food. A full phylogenetically informed comparative
analyses of the relative roles of octopaminergic and dopaminergic elements
of the reward system in the rewarding nature of sweet taste, nutrition and
social foraging across social and solitary species may provide a way to test
this hypothesis.
The examples available across insects and mammals indicate that evolu-
tionary modification of reward-directed behaviour has been important
in the evolution of new forms of social behaviour (O’Connell and
Hofmann, 2011). The adaptations have involved changes in stimuli or
behaviour that are activational of the reward system more than structural
changes in the circuitry of the reward system itself. In voles, for example,
the essential mesolimbic dopaminergic reward system is essentially the same
216 Eirik Søvik et al.

in non-monogamous and monogamous species: the key difference is in the
extent to which the reward system is activated by social recognition cues
(Donaldson and Young, 2008; Young and Wang, 2004). As has been dem-
onstrated by the comparison of honey bees and fruit flies, the essential
reward system may be quite similar across divergent insect lineages, but
the basis of behavioural differences between lineages may lie in the behav-
ioural modules that are linked to the reward system. The extensive changes
in gene regulation that have been linked to the evolution of eusociality
(Simola et al., 2013) may have been part of the mechanism by which an
ancestral reward system was “remodelled” in the process of social evolution.
5. CONCLUDING REMARKS
The honey bee and Drosophila are the two canonical model systems for
insect neuroscience research, but in many ways these two insects could not
be more different. They occur in divergent and derived lineages, and it is
important to consider to what extent the findings from one system can gen-
eralise to the other. The reward systems of the two species provide an infor-
mative point of comparison. Reward is involved in many different aspects of
behaviour, but despite marked differences in behaviour, ecology and evo-
lutionary history between honey bees and fruit flies the reward systems
are quite similar. Both involve networks of aminergic circuits to process dif-
ferent elements of reward. The honey bee literature has emphasised the role
of OA in reward, whereas the fruit fly literature is increasingly emphasising
specific dopaminergic circuits interacting with octopaminergic and pep-
tidergic circuits in reward processing. At the time of writing, this apparent
difference is more likely due to the different resolution of analysis that has
been possible in the two systems rather than a difference in biology. Neu-
rogenetic tools available for Drosophila have allowed extremely focused and
precise investigation of specific circuits. By comparison, the pharmacolog-
ical methods that have predominated the honey bee literature are coarser and
may struggle to differentiate the roles of OA and DA and/or specific circuits.
We propose an analysis of the possible role of DA in reward in honey bees as
a priority for research.
There remain some significant points of contrast between the fruit fly and
honey bee literature: most notably the findings that direct stimulation of the
octopaminergic neuron VUMmx1 is sufficient for reward learning in bees,
but direct stimulation of the extremely similar (and likely homologous) neu-
ron in flies, OA-VUMa2 did not have the same behavioural outcome.
217Insect Reward Systems: Comparing Flies and Bees

Whether this difference is due to a difference in the learning assays used
across the two systems or a genuine difference in the weighting of OA signals
between the two species is currently unclear.
A consideration of the role of the reward system in the evolution of new
forms of social behaviour would suggest that this can be achieved by evo-
lutionary modification of the stimuli or actions that activate the reward sys-
tem, rather than by direct modification of the reward system itself.
Consequently, we might expect the reward system to remain stable and rea-
sonably conserved across insect lineages, and to be a useful point of reference
for comparative neuroscience research.
ACKNOWLEDGEMENTS
We would like to thank Angelique Paulk for contributing images for Fig. 1 and Scott Waddell
for helpful discussions.
REFERENCES
Agarwal, M., Giannoni Guzma
´n, M., Morales-Matos, C., Del Valle Dı
´az, R.A.,
Abramson, C.I., Giray, T., 2011. Dopamine and octopamine influence avoidance learn-
ing of honey bees in a place preference assay. PLoS One 6, e25371.
Ament, S.A., Velarde, R.A., Kolodkin, M.H., Moyse, D., Robinson, G.E., 2011. Neuro-
peptide Y-like signalling and nutritionally mediated gene expression and behaviour in
the honey bee. Insect Mol. Biol. 20, 335–345.
Aso, Y., Siwanowicz, I., Bra
¨cker, L., Ito, K., Kitamoto, T., Tanimoto, H., 2010. Specific
dopaminergic neurons for the formation of labile aversive memory. Curr. Biol.
20, 1445–1451.
Aso, Y., Herb, A., Ogueta, M., Siwanowicz, I., Templier, T., Friedrich, A.B., Ito, K.,
Scholz, H., Tanimoto, H., 2012. Three dopamine pathways induce aversive odor mem-
ories with different stability. PLoS Genet. 8, e1002768.
Avargue
`s-Weber, A., Portelli, G., Benard, J., Dyer, A., Giurfa, M., 2010. Configural
processing enables discrimination and categorization of face-like stimuli in honeybees.
J. Exp. Biol. 213, 593–601.
Bailey, C., 1926. Epicurus: The Extant Remains. Clarendon Press, Oxford, UK.
Barron, A., Corbet, S., 1999. Preimaginal conditioning in Drosophila revisited. Anim. Behav.
58, 621–628.
Barron, A.B., Robinson, G.E., 2005. Selective modulation of task performance by
octopamine in honey bee (Apis mellifera) division of labour. J. Comp. Physiol.
A Neuroethol. Sens. Neural Behav. Physiol. 191, 659–668.
Barron, A.B., Schulz, D.J., Robinson, G.E., 2002. Octopamine modulates responsiveness to
foraging-related stimuli in honey bees (Apis mellifera). J. Comp. Physiol. A Neuroethol.
Sens. Neural Behav. Physiol. 188, 603–610.
Barron, A.B., Maleszka, R., Vander Meer, R.K., Robinson, G.E., 2007a. Octopamine mod-
ulates honey bee dance behavior. Proc. Natl. Acad. Sci. U.S.A. 104, 1703–1707.
Barron, A.B., Vander Meer, R.K., Maleszka, J., Robinson, G.E., Maleszka, R., 2007b.
Comparing injection, feeding and topical application methods for treatment of honey-
bees with octopamine. J. Insect Physiol. 53, 187–194.
Barron, A.B., Maleszka, R., Helliwell, P.G., Robinson, G.E., 2009. Effects of cocaine on
honey bee dance behaviour. J. Exp. Biol. 212, 163–168.
218 Eirik Søvik et al.

Barron, A.B., Søvik, E., Cornish, J.L., 2010. The roles of dopamine and related compounds
in reward-seeking behavior across animal phyla. Front. Behav. Neurosci. 4, 163.
Bath, D.E., Stowers, J.R., H€ormann, D., Poehlmann, A., Dickson, B.J., Straw, A.D., 2014.
FlyMAD: rapid thermogenetic control of neuronal activity in freely walking Drosophila.
Nat. Methods 11, 756–762.
Beggs, K.T., Tyndall, J.D.A., Mercer, A.R., 2011. Honey bee dopamine and octopamine
receptors linked to intracellular calcium signaling have a close phylogenetic and pharma-
cological relationship. PLoS One 6, e26809.
Ben-Shahar, Y., 2014. A piggyBac route to transgenic honeybees. Proc. Natl. Acad. Sci.
U.S.A. 111, 8708–8709.
Berridge, K.C., Kringelbach, M.L., 2008. Affective neuroscience of pleasure: reward in
humans and animals. Psychopharmacology (Berl) 199, 457–480.
Berridge, K.C., Valenstein, E.S., 1991. What psychological process mediates feeding evoked
by electrical stimulation of the lateral hypothalamus? Behav. Neurosci. 105, 3–14.
Berridge, K.C., Robinson, T.E., Aldridge, J.W., 2009. Dissecting components of reward:
“liking”, “wanting”, and learning. Curr. Opin. Pharmacol. 9, 65–73.
Beshel, J., Zhong, Y., 2013. Graded encoding of food odor value in the Drosophila brain.
J. Neurosci. 33, 15693–15704.
Bicker, G., Scha
¨fer, S., Kingan, T.G., 1985. Mushroom body feedback interneurones in the
honeybee show GABA-like immunoreactivity. Brain Res. 360, 394–397.
Bitterman, M.E., Menzel, R., Fietz, A., Scha
¨fer, S., 1983. Classical conditioning of proboscis
extension in honeybees (Apis mellifera). J. Comp. Psychol. 97, 107–119.
Bjordal, M., Arquier, N., Kniazeff, J., Pin, J.P., Le
´opold, P., 2014. Sensing of amino acids in a
dopaminergic circuitry promotes rejection of an incomplete diet in Drosophila. Cell
156, 510–521.
Borue, X., Condron, B., Venton, B.J., 2010. Both synthesis and reuptake are critical
for replenishing the releasable serotonin pool in Drosophila. J. Neurochem.
113, 188–199.
Brand, A.H., Perrimon, N., 1993. Targeted gene expression as a means of altering cell fates
and generating dominant phenotypes. Development 118, 401–415.
Bra
¨unig, P., 1991. Suboesophageal DUM neurons innervate the principal neuropiles of the
locust brain. Philos. Trans. R. Soc. B Biol. Sci. 332, 221–240.
Bra
¨unig, P., Burrows, M., 2004. Projection patterns of posterior dorsal unpaired median neu-
rons of the locust subesophageal ganglion. J. Comp. Neurol. 478, 164–175.
Breed, M., 1983. Nestmate recognition in honey bees. Anim. Behav. 31, 86–91.
Breed, M.D., Velthuis, H.H.W., Robinson, G.E., 1984. Do worker honey bees dis-
criminate among unrelated and related larval phenotypes. Ann. Entomol. Soc. Am.
77, 737–739.
Breed, M.D., Stiller, T.M., Moor, M.J., 1988. The ontogeny of kin discrimination cues in
the honey bee, Apis mellifera. Behav. Genet. 18, 439–448.
Breed, M., Welch, C., Cruz, R., 1994. Kin discrimination within honey bee (Apis mellifera)
colonies: an analysis of the evidence. Behav. Processes 33, 25–39.
Brown, M.R., Crim, J.W., Arata, R.C., Cai, H.N., Chun, C., Shen, P., 1999. Identification
of a Drosophila brain-gut peptide related to the neuropeptide Y family. Peptides
20, 1035–1042.
Burke, C.J., Waddell, S., 2011. Remembering nutrient quality of sugar in Drosophila. Curr.
Biol. 21, 746–750.
Burke, C.J., Huetteroth, W., Owald, D., Perisse, E., Krashes, M.J., Das, G., Gohl, D.,
Silies, M., Certel, S., Waddell, S., 2012. Layered reward signalling through octopamine
and dopamine in Drosophila. Nature 492, 433–437.
Busch, S., Tanimoto, H., 2010. Cellular configuration of single octopamine neurons in
Drosophila. J. Comp. Neurol. 518, 2355–2364.
219Insect Reward Systems: Comparing Flies and Bees

Busch, S., Selcho, M., Ito, K., Tanimoto, H., 2009. A map of octopaminergic neurons in the
Drosophila brain. J. Comp. Neurol. 513, 643–667.
Carr, D.B., Sesack, S.R., 2000. GABA-containing neurons in the rat ventral tegmental area
project to the prefrontal cortex. Synapse 38, 114–123.
Chapman, R.F., 2013. The Insects: Structure and Function. Cambridge University Press,
Cambridge, UK.
Christiansen, F., Zube, C., Andlauer, T.F.M., Wichmann, C., Fouquet, W., Owald, D.,
Mertel, S., Leiss, F., Tavosanis, G., Luna, A.J.F., et al., 2011. Presynapses in Kenyon cell
dendrites in the mushroom body calyx of Drosophila. J. Neurosci. 31, 9696–9707.
Claridge-Chang, A., Roorda, R.D., Vrontou, E., Sjulson, L., Li, H., Hirsh, J.,
Miesenb€ock, G., 2009. Writing memories with light-addressable reinforcement cir-
cuitry. Cell 139, 405–415.
Davis, R.L., 2005. Olfactory memory formation in Drosophila: from molecular to systems
neuroscience. Annu. Rev. Neurosci. 28, 275–304.
Denker, M., Finke, R., Schaupp, F., Gr€un, S., Menzel, R., 2010. Neural correlates of odor
learning in the honeybee antennal lobe. Eur. J. Neurosci. 31, 119–133.
Dietzl, G., Chen, D., Schnorrer, F., Su, K.-C., Barinova, Y., Fellner, M., Gasser, B.,
Kinsey, K., Oppel, S., Scheiblauer, S., et al., 2007. A genome-wide transgenic RNAi
library for conditional gene inactivation in Drosophila. Nature 448, 151–156.
Donaldson, Z.R., Young, L.J., 2008. Oxytocin, vasopressin, and the neurogenetics of soci-
ality. Science 322, 900–904.
Dreher, J.C., Tremblay, L., 2009. Handbook of Reward and Decision Making. Academic
Press, New York.
Ehmer, B., Gronenberg, W., 2002. Segregation of visual input to the mushroom bodies in
the honeybee (Apis mellifera). J. Comp. Neurol. 451, 362–373.
Erber, J., Kloppenburg, P., Scheidler, A., 1993. Neuromodulation by serotonin and
octopamine in the honeybee: behaviour, neuroanatomy and electrophysiology.
Experientia 49, 1073–1083.
Faber, T., Joerges, J., Menzel, R., 1999. Associative learning modifies neural representations
of odors in the insect brain. Nat. Neurosci. 2, 74–78.
Farooqui, T., Robinson, K., Vaessin, H., Smith, B.H., 2003. Modulation of early olfactory
processing by an octopaminergic reinforcement pathway in the honeybee. J. Neurosci.
23, 5370–5380.
Farooqui, T., Vaessin, H., Smith, B.H., 2004. Octopamine receptors in the honeybee (Apis
mellifera) brain and their disruption by RNA-mediated interference. J. Insect Physiol.
50, 701–713.
Ferguson, J.N., Young, L.J., Hearn, E.F., Matzuk, M.M., Insel, T.R., Winslow, J.T., 2000.
Social amnesia in mice lacking the oxytocin gene. Nat. Genet. 25, 284–288.
Ferguson, J.N., Aldag, J.M., Insel, T.R., Young, L.J., 2001. Oxytocin in the medial amygdala
is essential for social recognition in the mouse. J. Neurosci. 21, 8278–8285.
Fernandez, P.C., Locatelli, F.F., Person-Rennell, N., Deleo, G., Smith, B.H., 2009. Asso-
ciative conditioning tunes transient dynamics of early olfactory processing. J. Neurosci.
29, 10191–10202.
Fujita, M., Tanimura, T., 2011. Drosophila evaluates and learns the nutritional value of
sugars. Curr. Biol. 21, 751–755.
Galizia, C.G., 2014. Olfactory coding in the insect brain: data and conjectures. Eur. J. Neu-
rosci. 39, 1784–1795.
Galizia, C., Menzel, R., 2001. The role of glomeruli in the neural representation of odours:
results from optical recording studies. J. Insect Physiol. 47, 115–130.
Gardner, E.L., 2005. Endocannabinoid signaling system and brain reward: emphasis on dopa-
mine. Pharmacol. Biochem. Behav. 81, 263–284.
220 Eirik Søvik et al.

Gendron, C.M., Kuo, T.-H., Harvanek, Z.M., Chung, B.Y., Yew, J.Y., Dierick, H.A.,
Pletcher, S.D., 2014. Drosophila life span and physiology are modulated by sexual per-
ception and reward. Science 343, 544–548.
Giurfa, M., 2013. Cognition with few neurons: higher-order learning in insects. Trends
Neurosci. 36, 285–294.
Grimaldi, D., Engel, M.S., 2005. Evolution of the Insects. Cambridge University Press,
Cambridge, UK.
Gr€unewald, B., 1999a. Physiological properties and response modulations of mushroom
body feedback neurons during olfactory learning in the honeybee, Apis mellifera.
J. Comp. Physiol. A 185, 565–576.
Gr€unewald, B., 1999b. Morphology of feedback neurons in the mushroom body of the hon-
eybee, Apis mellifera. J. Comp. Neurol. 404, 114–126.
Hamada, F.N., Rosenzweig, M., Kang, K., Pulver, S.R., Ghezzi, A., Jegla, T.J., Garrity, P.a.,
2008. An internal thermal sensor controlling temperature preference in Drosophila.
Nature 454, 217–220.
Hammer, M., 1993. An identified neuron mediates the unconditioned stimulus in associative
olfactory learning in honeybees. Nature 366, 59–63.
Hammer, M., Menzel, R., 1995. Learning and memory in the honeybee. J. Neurosci.
15, 1617–1630.
Hammer, M., Menzel, R., 1998. Multiple sites of associative odor learning as revealed by
local brain microinjections of octopamine in honeybees. Learn. Mem. 5, 146–156.
Han, K.A., Millar, N.S., Davis, R.L., 1998. A novel octopamine receptor with preferential
expression in Drosophila mushroom bodies. J. Neurosci. 18, 3650–3658.
Heisenberg, M., 2003. Mushroom body memoir: from maps to models. Nat. Rev. Neurosci.
4, 266–275.
Heisenberg, M., Borst, A., Wagner, S., Byers, D., 1985. Drosophila mushroom body mutants
are deficient in olfactory learning. J. Neurogenet. 2, 1–30.
Honjo, K., Furukubo-Tokunaga, K., 2009. Distinctive neuronal networks and biochemical
pathways for appetitive and aversive memory in Drosophila larvae. J. Neurosci. 29, 852–862.
Hu, A., Zhang, W., Wang, Z., 2010. Functional feedback from mushroom bodies to anten-
nal lobes in the Drosophila olfactory pathway. Proc. Natl. Acad. Sci. U.S.A.
107, 10262–10267.
Ikemoto, S., 2010. Brain reward circuitry beyond the mesolimbic dopamine system: a neu-
robiological theory. Neurosci. Biobehav. Rev. 35, 129–150.
Kenyon, F., 1896. The meaning and structure of the so-called “mushroom bodies” of the
hexapod brain. Am. Nat. 30, 643–650.
Kim, Y.-C., Lee, H.-G., Han, K.-A., 2007. D1 dopamine receptor dDA1 is required in the
mushroom body neurons for aversive and appetitive learning in Drosophila. J. Neurosci.
27, 7640–7647.
Kitamoto, T., 2001. Conditional modification of behavior in Drosophila by targeted expres-
sion of a temperature-sensitive shibire allele in defined neurons. J. Neurobiol. 47, 81–92.
Knaden, M., Strutz, A., Ahsan, J., Sachse, S., Hansson, B.S., 2012. Spatial representation of
odorant valence in an insect brain. Cell Rep. 1, 392–399.
Knapek, S., Kahsai, L., Winther, A.M.E., Tanimoto, H., Na
¨ssel, D.R., 2013. Short neuro-
peptide F acts as a functional neuromodulator for olfactory memory in Kenyon cells of
Drosophila mushroom bodies. J. Neurosci. 33, 5340–5345.
Kranz, G.S., Kasper, S., Lanzenberger, R., 2010. Reward and the serotonergic system.
Neuroscience 166, 1023–1035.
Krashes, M.J., DasGupta, S., Vreede, A., White, B., Armstrong, J.D., Waddell, S., 2009.
A neural circuit mechanism integrating motivational state with memory expression in
Drosophila. Cell 139, 416–427.
221Insect Reward Systems: Comparing Flies and Bees

Kreissl, S., Eichm€uller, S., Bicker, G., Rapus, J., Eckert, M., Eichm€uller, S., 1994.
Octopamine-like immunoreactivity in the brain and subesophageal ganglion of the hon-
eybee. J. Comp. Neurol. 348, 583–595.
Kuwabara, M., 1957. Bildung des bedingten Reflexes von Pavlovs Typus bei der
Honigbiene, Apis mellifica. J. Fac. Sci. Hokaido Univ. 8, 458–464.
Lee, K.P., Simpson, S.J., Clissold, F.J., Brooks, R., Ballard, J.W.O., Taylor, P.W., Soran, N.,
Raubenheimer, D., 2008. Lifespan and reproduction in Drosophila: new insights from
nutritional geometry. Proc. Natl. Acad. Sci. U.S.A. 105, 2498–2503.
Leknes, S., Tracey, I., 2008. A common neurobiology for pain and pleasure. Nat. Rev. Neu-
rosci. 9, 314–320.
Liu, C., Plac¸ais, P.-Y., Yamagata, N., Pfeiffer, B.D., Aso, Y., Friedrich, A.B.,
Siwanowicz, I., Rubin, G.M., Preat, T., Tanimoto, H., 2012. A subset of dopamine
neurons signals reward for odour memory in Drosophila. Nature 488, 512–516.
Locatelli, F.F., Fernandez, P.C., Villareal, F., Muezzinoglu, K., Huerta, R., Galizia, C.G.,
Smith, B.H., 2013. Nonassociative plasticity alters competitive interactions among mix-
ture components in early olfactory processing. Eur. J. Neurosci. 37, 63–79.
Loeb, J., 1918. Forced Movements, Tropisms, and Animal Conduct. Dover, New York.
Long, T.F., Murdock, L.L., 1983. Stimulation of blowfly feeding behavior by
octopaminergic drugs. Proc. Natl. Acad. Sci. U.S.A. 80, 4159–4163.
Mahler, S.V., Smith, K.S., Berridge, K.C., 2007. Endocannabinoid hedonic hotspot for sen-
sory pleasure: anandamide in nucleus accumbens shell enhances “liking” of a sweet
reward. Neuropsychopharmacology 32, 2267–2278.
McQuillan, H.J., Nakagawa, S., Mercer, A.R., 2012. Mushroom bodies of the honeybee
brain show cell population-specific plasticity in expression of amine-receptor genes.
Learn. Mem. 19, 151–158.
Menzel, R., 2001. Searching for the memory trace in a mini-brain, the honeybee. Learn.
Mem. 8, 53–62.
Menzel, R., Manz, G., 2005. Neural plasticity of mushroom body-extrinsic neurons in the
honeybee brain. J. Exp. Biol. 208, 4317–4332.
Menzel, R., Erber, J., Masuhr, T., 1974. Learning and memory in the honeybee.
In: Browne, L.B. (Ed.), Experimental Analysis of Insect Behaviour. Springer-Verlag,
Berlin, pp. 195–217.
Menzel, R., Heyne, A., Kinzel, C., Gerber, B., Fiala, A., 1999. Pharmacological dissociation
between the reinforcing, sensitizing, and response-releasing functions of reward in hon-
eybee classical conditioning. Behav. Neurosci. 113, 744–754.
Mercer, A., Menzel, R., 1982. The effects of biogenic amines on conditioned and uncon-
ditioned responses to olfactory stimuli in the honey bee Apis mellifera. J. Comp. Physiol.
A Neuroethol. Sens. Neural Behav. Physiol. 145, 363–368.
Miyamoto, T., Slone, J., Song, X., Amrein, H., 2012. A fructose receptor functions as a
nutrient sensor in the Drosophila brain. Cell 151, 1113–1125.
Mizunami, M., Unoki, S., Mori, Y., Hirashima, D., Hatano, A., Matsumoto, Y., 2009.
Roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory
recall in an insect. BMC Biol. 7, 46.
Mobbs, P., 1982. The brain of the honey bee Apis mellifera. I. The connections and spatial
organization of the mushroom bodies. Philos. Trans. R. Soc. B Biol. Sci. 298, 309–354.
Mustard, J.A., Beggs, K.T., Mercer, A.R., 2005. Molecular biology of the invertebrate dopa-
mine receptors. Arch. Insect Biochem. Physiol. 59, 103–117.
Nagel, G., Szellas, T., Huhn, W., Kateriya, S., Adeishvili, N., Berthold, P., Ollig, D.,
Hegemann, P., Bamberg, E., 2003. Channelrhodopsin-2, a directly light-gated
cation-selective membrane channel. Proc. Natl. Acad. Sci. U.S.A. 100, 13940–13945.
O’Connell, L.A., Hofmann, H.A., 2011. The vertebrate mesolimbic reward system and
social behavior network: a comparative synthesis. J. Comp. Neurol. 519, 3599–3639.
222 Eirik Søvik et al.

Okada, R., Rybak, J., Manz, G., Menzel, R., 2007. Learning-related plasticity in PE1 and
other mushroom body-extrinsic neurons in the honeybee brain. J. Neurosci.
27, 11736–11747.
Omelchenko, N., Sesack, S.R., 2009. Ultrastructural analysis of local collaterals of rat ventral
tegmental area neurons: GABA phenotype and synapses onto dopamine and GABA cells.
Synapse 63, 895–906.
Page Jr., R.E., Breed, M., 1987. Kin recognition in social bees. Trends Ecol. Evol.
2, 272–275.
Parnas, M., Lin, A.C., Huetteroth, W., Miesenb€ock, G., 2013. Odor discrimination in Dro-
sophila: from neural population codes to behavior. Neuron 79, 932–944.
Paulk, A.C., Stacey, J.A., Pearson, T.W.J., Taylor, G.J., Moore, R.J.D., Srinivasan, M.V.,
van Swinderen, B., 2014. Selective attention in the honeybee optic lobes precedes
behavioral choices. Proc. Natl. Acad. Sci. U.S.A. 111, 5006–5011.
Pavlov, I.P., 1927. Conditioned Reflexes. Oxford University Press, London, UK.
Pecin
˜a, S., Berridge, K.C., 2005. Hedonic hot spot in nucleus accumbens shell: where
do mu-opioids cause increased hedonic impact of sweetness? J. Neurosci.
25, 11777–11786.
Perisse, E., Burke, C., Huetteroth, W., Waddell, S., 2013. Shocking revelations and saccha-
rin sweetness in the study of Drosophila olfactory memory. Curr. Biol. 23, R752–R763.
Perry, C.J., Barron, A.B., 2013. Neural mechanisms of reward in insects. Annu. Rev.
Entomol. 58, 543–562.
Quinn, W., Harris, W., Benzer, S., 1974. Conditioned behavior in Drosophila melanogaster.
Proc. Natl. Acad. Sci. U.S.A. 71, 708–712.
Rath, L., Giovanni Galizia, C., Szyszka, P., 2011. Multiple memory traces after associative
learning in the honey bee antennal lobe. Eur. J. Neurosci. 34, 352–360.
Rein, J., Mustard, J.A., Strauch, M., Smith, B.H., Galizia, C.G., 2013. Octopamine mod-
ulates activity of neural networks in the honey bee antennal lobe. J. Comp. Physiol.
A Neuroethol. Sens. Neural Behav. Physiol. 199, 947–962.
Root, C.M., Ko, K.I., Jafari, A., Wang, J.W., 2011. Presynaptic facilitation by neuropeptide
signaling mediates odor-driven food search. Cell 145, 133–144.
Rybak, J., Menzel, R., 1998. Integrative properties of the Pe1 neuron, a unique mushroom
body output neuron. Learn. Mem. 5, 133–145.
Scheiner, R., Pl€uckhahn, S., Oney, B., Blenau, W., Erber, J., 2002. Behaviouralpharmacology
of octopamine, tyramine and dopamine in honey bees. Behav. Brain Res. 136, 545–553.
Scheiner, R., Baumann, A., Blenau, W., 2006. Aminergic control and modulation of hon-
eybee behaviour. Curr. Neuropharmacol. 4, 259–276.
Scheiner, R., Steinbach, A., Claßen, G., Strudthoff, N., Scholz, H., 2014. Octopamine indi-
rectly affects proboscis extension response habituation in Drosophila melanogaster by con-
trolling sucrose responsiveness. J. Insect Physiol. 69C, 107–117.
Schroll, C., Riemensperger, T., Bucher, D., Ehmer, J., Vo, T., Erbguth, K., Gerber, B.,
Hendel, T., Nagel, G., Buchner, E., et al., 2006. Report light-induced activation of dis-
tinct modulatory neurons triggers appetitive or aversive learning in Drosophila larvae.
Curr. Biol. 16, 1741–1747.
Schr€oter, U., Malun, D., Menzel, R., 2007. Innervation pattern of suboesophageal ventral
unpaired median neurones in the honeybee brain. Cell Tissue Res. 327, 647–667.
Schulte, C., Theilenberg, E., M€uller-Borg, M., Gempe, T., Beye, M., 2014. Highly efficient
integration and expression of piggyBac-derived cassettes in the honeybee (Apis mellifera).
Proc. Natl. Acad. Sci. U.S.A. 111, 9003–9008.
Schultz, W., 1997. Dopamine neurons and their role in reward mechanisms. Curr. Opin.
Neurobiol. 7, 191–197.
Schultz, W., 2002. Getting formal with dopamine and reward. Neuron 36, 241–263.
Schultz, W., 2007. Behavioral dopamine signals. Trends Neurosci. 30, 203–210.
223Insect Reward Systems: Comparing Flies and Bees

Schultz, W., 2010. Dopamine signals for reward value and risk: basic and recent data. Behav.
Brain Funct. 6, 24.
Schultz, W., Dayan, P., Montague, P.R., 1997. A neural substrate of prediction and reward.
Science 275, 1593–1599.
Schulz, D.J., Barron, A.B., Robinson, G.E., 2002. A role for octopamine in honey bee divi-
sion of labor. Brain Behav. Evol. 60, 350–359.
Schwaerzel, M., Monastirioti, M., Scholz, H., Friggi-Grelin, F., Birman, S., Heisenberg, M.,
2003. Dopamine and octopamine differentiate between aversive and appetitive
olfactory memories in Drosophila. J. Neurosci. 23, 10495–10502.
Seeley, T.D., 1989. The bee colony as a superorganism. Am. Sci. 77, 546–553.
Seeley, T.D., 1995. The Wisdom of the Hive. Harvard University Press, Cambridge, MA.
Selcho, M., Pauls, D., Han, K.-A., Stocker, R.F., Thum, A.S., 2009. The role of dopamine
in Drosophila larval classical olfactory conditioning. PLoS One 4, e5897.
Shimada, T., Kato, K., Kamikouchi, A., Ito, K., 2005. Analysis of the distribution of the brain
cells of the fruit fly by an automatic cell counting algorithm. Phys. A Stat. Mech. Appl.
350, 144–149.
Shohat-Ophir, G., Kaun, K.R., Azanchi, R., Heberlein, U., 2012. Sexual deprivation
increases ethanol intake in Drosophila. Science 335, 1351–1355.
Simola, D.F., Wissler, L., Donahue, G., Waterhouse, R.M., Helmkampf, M., Roux, J.,
Nygaard, S., Glastad, K.M., Hagen, D.E., Viljakainen, L., et al., 2013. Social insect
genomes exhibit dramatic evolution in gene composition and regulation while
preserving regulatory features linked to sociality. Genome Res. 23, 1235–1247.
Sinakevitch, I., Strausfeld, N.J., 2006. Comparison of octopamine-like immunoreactivity in
the brains of the fruit fly and blow fly. J. Comp. Neurol. 494, 460–475.
Sinakevitch, I., Niwa, M., Strausfeld, N.J., 2005. Octopamine-like immunoreactivity in
the honey bee and cockroach: comparable organization in the brain and subesophageal
ganglion. J. Comp. Neurol. 488, 233–254.
Smith, K.S., Berridge, K.C., 2005. The ventral pallidum and hedonic reward: neurochemical
maps of sucrose “liking” and food intake. J. Neurosci. 25, 8637–8649.
Smith, K.S., Mahler, S.V., Pecin
˜a, S., Berridge, K.C., 2007. Hedonic hotspots: generating
sensory pleasure in the brain. In: Kringelbach, M.L., Berridge, K.C. (Eds.), Pleasures of
the Brain. Oxford University Press, New York, pp. 27–49.
Smith, K.S., Tindell, A.J., Aldridge, J.W., Berridge, K.C., 2009. Ventral pallidum roles in
reward and motivation. Behav. Brain Res. 196, 155–167.
Søvik, E., 2013. Reward Processing and Responses to Drugs of Abuse in the Honey Bee,
Apis mellifera. Macquarie University, Australia.
Søvik, E., Barron, A.B., 2013. Invertebrate models in addiction research. Brain Behav. Evol.
82, 153–165.
Stocker, R.F., 1994. The organization of the chemosensory system in Drosophila melanogaster:
a review. Cell Tissue Res. 275, 3–26.
Strausfeld, N.J., 2002. Organization of the honey bee mushroom body: representation of the
calyx within the vertical and gamma lobes. J. Comp. Neurol. 450, 4–33.
Strube-Bloss, M.F., Nawrot, M.P., Menzel, R., 2011. Mushroom body output neurons
encode odor-reward associations. J. Neurosci. 31, 3129–3140.
Szyszka, P., Ditzen, M., Galkin, A., Galizia, C.G., Menzel, R., 2005. Sparsening and tem-
poral sharpening of olfactory representations in the honeybee mushroom bodies.
J. Neurophysiol. 94, 3303–3313.
Szyszka, P., Galkin, A., Menzel, R., 2008. Associative and non-associative plasticity in
Kenyon cells of the honeybee mushroom body. Front. Syst. Neurosci. 2, 3.
Tanimoto, H., Heisenberg, M., Gerber, B., 2004. Event timing turns punishment to reward.
Nature 430, 983.
224 Eirik Søvik et al.

Technau, G., Heisenberg, M., 1982. Neural reorganization during metamorphosis of the
corpora pedunculata in Drosophila melanogaster. Nature 295, 405–407.
Tempel, B.L., Bonini, N., Dawson, D.R., Quinn, W.G., 1983. Reward learning in normal
and mutant Drosophila. Proc. Natl. Acad. Sci. U.S.A. 80, 1482–1486.
Tempel, B.L., Livingstone, M.S., Quinn, W.G., 1984. Mutations in the dopa decarboxylase
gene affect learning in Drosophila. Proc. Natl. Acad. Sci. U.S.A. 81, 3577–3581.
Thum, A.S., Jenett, A., Ito, K., Heisenberg, M., Tanimoto, H., 2007. Multiple memory
traces for olfactory reward learning in Drosophila. J. Neurosci. 27, 11132–11138.
Tibbetts, E.A., 2002. Visual signals of individual identity in the wasp Polistes fuscatus. Proc. R.
Soc. B Biol. Sci. 269, 1423–1428.
Tinbergen, N., 1951. The Study of Instinct. Oxford University Press, New York.
Truman, J.W., Riddiford, L.M., 1999. The origins of insect metamorphosis. Nature
401, 447–452.
Unoki, S., Matsumoto, Y., Mizunami, M., 2005. Participation of octopaminergic reward
system and dopaminergic punishment system in insect olfactory learning revealed by
pharmacological study. Eur. J. Neurosci. 22, 1409–1416.
Unoki, S., Matsumoto, Y., Mizunami, M., 2006. Roles of octopaminergic and dopaminergic
neurons in mediating reward and punishment signals in insect visual learning. Eur. J.
Neurosci. 24, 2031–2038.
Van Bockstaele, E.J., Pickel, V.M., 1995. GABA-containing neurons in the ventral
tegmental area project to the nucleus accumbens in rat brain. Brain Res.
682, 215–221.
Van Ree, J.M., Niesink, R.J., Van Wolfswinkel, L., Ramsey, N.F., Kornet, M.M., Van
Furth, W.R., Vanderschuren, L.J., Gerrits, M.A., Van den Berg, C.L., 2000. Endoge-
nous opioids and reward. Eur. J. Pharmacol. 405, 89–101.
Vergoz, V., Roussel, E., Sandoz, J.-C., Giurfa, M., 2007. Aversive learning in honey-
bees revealed by the olfactory conditioning of the sting extension reflex. PLoS One
2, e288.
Vickrey, T.L., Condron, B., Venton, B.J., 2009. Detection of endogenous dopamine
changes in Drosophila melanogaster using fast-scan cyclic voltammetry. Anal. Chem.
81, 9306–9313.
Vlachou, S., Markou, A., 2010. GABA
B
receptors in reward processes. Adv. Pharmacol.
58, 315–371.
Vogt, K., Schnaitmann, C., Dylla, K.V., Knapek, S., Aso, Y., Rubin, G.M., Tanimoto, H.,
2014. Shared mushroom body circuits operate visual and olfactory memories in Drosoph-
ila. eLife 3, e02395.
von Frisch, K., 1914. Der farbensinn und Formensinn der Biene. Zool. Jahrb. Abt. Allg.
Zool. Physiol. Tiere 35, 1–188.
Wang, Z., Singhvi, A., Kong, P., Scott, K., 2004. Taste representations in the Drosophila
brain. Cell 117, 981–991.
Weiner, J., 1999. Time, Love, Memory: A Great Biologist and His Quest for the Origins of
Behavior. Knopf, New York.
Williams, J.R., Catania, K.C., Carter, C.S., 1992. Development of partner preferences in
female prairie voles (Microtus ochrogaster): the role of social and sexual experience. Horm.
Behav. 26, 339–349.
Winston, M.L., 1987. The Biology of the Honey Bee. Harvard University Press, Cambridge,
MA.
Wise, R.A., Rompre, P.P., 1989. Brain dopamine and reward. Annu. Rev. Psychol.
40, 191–225.
Witth€oft, W., 1967. Absolute Anzahl und Verteilung der Zellen im Hirn der Honigbiene.
Zoomorphology 61, 160–184.
225Insect Reward Systems: Comparing Flies and Bees

Wolf, R., Wittig, T., Liu, L., Wustmann, G., Eyding, D., Heisenberg, M., 1998. Drosophila
mushroom bodies are dispensable for visual, tactile, and motor learning. Learn. Mem.
5, 166–178.
Wright, G.A., Mustard, J.A., Simcock, N.K., Ross-Taylor, A.A.R., McNicholas, L.D.,
Popescu, A., Marion-Poll, F., 2010. Parallel reinforcement pathways for conditioned
food aversions in the honeybee. Curr. Biol. 20, 2234–2240.
Yarali, A., Gerber, B., 2010. A neurogenetic dissociation between punishment-, reward-,
and relief-learning in Drosophila. Front. Behav. Neurosci. 4, 189.
Young, L.J., Wang, Z., 2004. The neurobiology of pair bonding. Nat. Neurosci.
7, 1048–1054.
Young, L.J., Murphy Young, A.Z., Hammock, E.A.D., 2005. Anatomy and neurochemistry
of the pair bond. J. Comp. Neurol. 493, 51–57.
226 Eirik Søvik et al.