Content uploaded by Barbara J Knowlton
Author content
All content in this area was uploaded by Barbara J Knowlton on Jan 21, 2022
Content may be subject to copyright.
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
Annual Review of Psychology
Memory and Reward-Based
Learning: A Value-Directed
Remembering Perspective
Barbara J. Knowlton and Alan D. Castel
Department of Psychology, University of California, Los Angeles, California 90095, USA;
email: knowlton@psych.ucla.edu
Annu. Rev. Psychol. 2022. 73:15.1–15.28
The Annual Review of Psychology is online at
psych.annualreviews.org
https://doi.org/10.1146/annurev-psych-032921-
050951
Copyright © 2022 by Annual Reviews.
All rights reserved
Keywords
aging, encoding, episodic memory, metacognition, reward
Abstract
The ability to prioritize valuable information is critical for the efcient use
of memory in daily life. When information is important, we engage more
effective encoding mechanisms that can better support retrieval. Here, we
describe a dual-mechanism framework of value-directed remembering in
which both strategic and automatic processes lead to differential encoding
of valuable information. Strategic processes rely on metacognitive awareness
of effective deep encoding strategies that allow younger and healthy older
adults to selectively remember important information. In contrast, some
high-value information may also be encoded automatically in the absence of
intention to remember, but this may be more impaired in older age. These
different mechanisms are subserved by different neural substrates, with left-
hemisphere semantic processing regions active during the strategic encoding
of high-value items, and automatic enhancement of encoding of high-value
items may be supported by activation of midbrain dopaminergic projections
to the hippocampal region.
15.1
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
Contents
1. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15.2
2. STRATEGIC AND AUTOMATIC EFFECTS OF VALUE ON MEMORY . . . 15.3
3. STRATEGIC USEOF VALUETO ENHANCEMEMORY.................. 15.5
3.1. Self-Regulated Learning................................................. 15.6
3.2. Neural Mechanisms of Strategic Learning of Valuable Information . . . . . . . . 15.6
4. AUTOMATICEFFECTS OF VALUEONMEMORY ....................... 15.9
4.1. Interactions Between Reward Circuitry and the Hippocampus ............. 15.9
4.2. TheRoleofDopamine..................................................15.10
5. THE DEVELOPMENT OF VALUE-DIRECTED REMEMBERING . . . . . . . 15.13
6. MOTIVATED MEMORY AND AGING: AN ENHANCED
FOCUSONVALUE?.......................................................15.15
7. METACOGNITION GUIDING VALUE-DIRECTED REMEMBERING . . .15.17
8. BINDINGANDSPATIALMEMORY.......................................15.19
9. CURIOSITYANDREWARD-BASEDLEARNING.........................15.20
10. FUTUREDIRECTIONS ................................................... 15.21
11. SUMMARYANDCONCLUSIONS.........................................15.21
12. APPENDIX.................................................................15.22
1. INTRODUCTION
In daily life, we are confronted with a plethora of information. Some of this information is impor-
tant for us to remember, whereas other information is of little consequence and should be forgot-
ten. For example, imagine you are attending a conference and meeting dozens of colleagues,a few
of whom may be relevant to your current project. It would be important to remember the names
of the relevant colleagues, even if many names were forgotten. Selectivity is critical for memory to
be adaptive, as memory for irrelevant information may interfere with retrieving valuable informa-
tion. At a doctor’s visit, when the possible side effects of a new medication are discussed, retrieving
less important side effects should not interfere with memory for a side effect that demands imme-
diate medical attention. In this review, we explore the processes underlying our ability to prioritize
information in memory and our current knowledge of the underlying mechanisms. We also dis-
cuss how memory selectivity is affected in different populations and the factors that may serve to
enhance or impair the ability to preferentially remember valuable information in these groups.
We specically focus on older adults, often characterized as those over the age of 65, in whom
memory selectivity can compensate for reductions in memory capacity.
Memory selectivity for valuable information can be accomplished by the selective encoding
and consolidation of this information or its preferential retrieval at time of test. Individuals may
prioritize retrieving valuable information, for example, by retrieving it initially to avoid output
interference effects (Tulving & Arbuckle 1966). Value also appears to inuence search processes
in retrieval (Stefanidi et al. 2018). This retrieval prioritization depends on the reliable and robust
encoding of valuable information. Thus, value-directed remembering primarily involves differ-
entially encoding high-value information, which renders this information more accessible under
a range of retrieval conditions. Whereas much research in cognitive psychology and cognitive
neuroscience has focused on the neural and behavioral mechanisms of effective memory encod-
ing, the importance of selectivity of encoding has not received as much attention despite its clear
15.2 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
functional role. In the following sections, we review existing literature from different domains that
focuses on how valuable information is preferentially encoded.
Differential encoding of valuable information has heretofore been conceptualized in two dif-
ferent ways. First, people may strategically focus on information deemed important and engage
in deeper semantic processing of this information (Cohen et al. 2017). If you are in a new city
and discover a restaurant that was excellent, you may make a mental note of it so you can return
on your next visit. Making a mental note may include picturing the restaurant and its name or
associating the name with some other knowledge that you have. You may also practice retrieving
the name later if you write it down or tell a friend about it. Each of these actions would strengthen
memory for the restaurant and depend on your metamemory abilities. In other words, you have
some awareness of how to effectively encode information (imagery, semantic association, retrieval
practice) and use these encoding methods to remember this valuable piece of information. As de-
scribed in Section 3, there is a rich literature on metacognition, namely on how people regulate
their learning and shift to deeper encoding strategies based on experience, indicating that people
can apply more effective encoding strategies when information is deemed important (Dunlosky
1998, Hertzog et al. 2008). Thus, it is likely that a substantial component of value-directed re-
membering involves the conscious application of effective encoding methods.
The second mechanism by which value enhances memory appears to be more automatic. In-
formation that is rewarding is salient and preferentially remembered (for a review, see Schultz
2015). Following from the previous example, you may remember well the excellent restaurant in
the new city without any motivation to do so. Even if you have no plans to return to the new city,
the pleasant food and ambiance likely led to a strong memory for the event. This more automatic
effect of reward on memory is often thought of as arising from prediction error (den Ouden et al.
2012). When the outcome exceeds expectation, memory is strengthened. If the restaurant is sur-
prisingly good, your memory for it would be enhanced by this mechanism compared with your
memory for a restaurant of expected quality. In fact, even arbitrary stimuli that are proximal to
rewards are strengthened in memory (Braun et al. 2018). There has been extensive study of the
neural mechanisms of prediction error with a focus on the role of the dopamine signal (Glimscher
2011). Prediction error also features prominently as a key mechanism in learning models such as
the Rescorla–Wagner model (Rescorla & Wagner 1972). In Section 4 we review evidence that
reward can automatically strengthen memory via a prediction error mechanism and that impair-
ments in the frontostriatal representation of prediction error in older age (Samanez-Larkin et al.
2014) may impact this type of value-directed remembering in older adults.
The idea of dual mechanisms of reward was proposed in Bijleveld et al. (2012), in which reward
has initial, unconscious effects that facilitate performance, with later effects enabling strategic de-
cision making based on reward experience. Here, we apply this general idea specically to mem-
ory encoding. Acknowledging that there are two distinct mechanisms by which value enhances
memory allows us to better understand the conditions that affect the different mechanisms. For
example, strategic mechanisms that rely on metacognition may be less effective when resources
or motivation is low. Automatic effects of value may not be as effective if to-be-learned valuable
information does not generate a strong prediction error signal. Furthermore, positing these dis-
tinct mechanisms claries ndings in the literature and provides insights into different ways to
enhance value-directed remembering.
2. STRATEGIC AND AUTOMATIC EFFECTS OF VALUE ON MEMORY
One important factor in whether people engage in more strategic encoding of high-value items
is the form of the memory test. In value-directed remembering procedures often used in the
www.annualreviews.org •Memory and Reward-Based Learning 15.3
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
table 5
uncle 9
apple 2
pilot 6
berry 11
cabin 1
skate 7
cheek 12
fence 3
straw 8
petal 10
drain 4
Time
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1 2 3 4 5 6 7 8 9 10 11 12
Point value of word
Probability of recall
ab
Young
Old
Figure 1
(a) The selectivity task method often used in value-directed remembering, in which a series of 12 words are
presented sequentially and are randomly paired with 12 unique point values. Participants are told to recall as
many words as they can to maximize their score, which is the sum of the point values paired with the recalled
words. (b) Findings from the selectivity task plotted in terms of the probability of recall as a function of point
value for younger and older adults. Figure adapted from Castel et al. (2012).
laboratory, participants study successive lists of words in which each word is associated with a
point value. At the end of each list, participants try to recall as many words as possible and receive
the total point value of the recalled words (see Figure 1a). Across the rst few lists, participants
become more selective as they realize that they are able to recall only a limited number of items per
list (see Figure 1b), and focus on encoding only the highest-value items (Castel 2008). In this pro-
cedure, learning and recalling low-value information can reduce the probability of recalling high-
value information given the limited capacity of free recall. Thus, participants may try to direct
encoding to high-value items and try to avoid encoding low-value items. Effective performance,
therefore, relies on differential encoding strategies. Value can also benet memory when tested
through recognition. In this procedure, participants are presented with items associated with dif-
ferent point values and are told they will earn those points if they later recognize them at test
(Elliot & Brewer 2019, Hennessee et al. 2019a). Participants are also often told that they will lose
points for false alarms to prevent participants from adopting a strategy by which they call all items
old. Here, participants do not have successive lists from which to practice and they may not expe-
rience how differential encoding relates to performance. In addition, the amount of information
one can hold in memory (capacity limitations) does not come into play to the same extent as with a
free recall test, so participants may not feel the need to ignore less-valuable items. Under these cir-
cumstances, more-valuable items are recognized more frequently than lower-value items, though
such effects of value tend to be smaller than those for successive recall procedures. This difference
may reect a reduced contribution of differential strategic encoding and a relatively greater con-
tribution of automatic effects of value on encoding. These results indicate that the effects of value
on memory are apparent regardless of the manner of the test. However, strategic and automatic
effects of value may differentially contribute to memory depending on how it is measured.
Another important question is how value affects the quality of memory. That is, does value
specically enhance the episodic quality of a memory or does value strengthen memory overall,
leading to an increased sense of familiarity or the gist, a less precise, generic representation of the
15.4 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
memory? Here, we have some evidence that strategic and automatic effects of value may differen-
tially impact memory quality. Strengthening memory via deeper semantic encoding can enhance
both recollection (a detailed contextually rich memory) and familiarity (more general gist mem-
ory) for those items; thinking about the meaning of an item may create a robust episode but would
also strengthen semantic knowledge of the item (Carr et al. 2015, Yonelinas 2002). Thus, it may
be the case that when participants strategically focus on encoding high-value items, they generally
strengthen the underlying memory, reected as both better episodic memory and a greater sense
of familiarity for the items when later encountered.
In contrast, much of the work focusing on automatic effects of value points to a specic episodic
memory enhancement (Gruber et al. 2016). Items associated with high rewards are remembered
along with source information, such as the background of the item, with no enhancement of recog-
nition absent these source details or increased feelings of familiarity for them. Neuroimaging data
on the enhancing role of reward in memory have implicated dopaminergic projections into the
hippocampus as subserving this effect (Adcock et al. 2006). On the basis of evidence for a specic
role of the hippocampus in episodic memory,these results suggest that automatic effects of reward
may primarily enhance an episodic memory for items associated with reward (Moscovitch et al.
2016). Other behavioral effects are broadly consistent with this idea. When participants perform
initial free recall tests on material with different values, they subsequently show increases in both
recollection and familiarity of high-value items, suggesting that the participants are informed by
their prior performance and are engaged in deeper semantic encoding that is selective for high-
value items (Cohen et al. 2017). When participants do not have experience with free recall and
they seek to then study all high- and low-value items for a recognition test, the effects of value
tend to be conned to measures of enhanced recollection in episodic memory (Elliot et al. 2020a,
Hennessee et al. 2017). These results hint at a possible dissociation between the mechanisms of
strategic and automatic effects of value. In Sections 3 and 4 we describe these different mechanisms
and their potential neural substrates.
3. STRATEGIC USE OF VALUE TO ENHANCE MEMORY
When people are aware that some information is important or will be valuable later, they engage
in more effective processing of this information to increase the probability of subsequent memory.
When possible, people may ofoad information by writing down notes or taking a photograph
of information to be remembered. However, people also have some awareness of how to encode
information more effectively into memory (Ariel et al. 2009). Although people may not generally
be aware of highly effective encoding strategies (e.g., Kornell & Bjork 2007), they nevertheless
use experience to improve their memory performance on specic tasks by applying more effective
encoding strategies (Storm et al. 2016). These strategies may include spending more time on
information deemed valuable, using mental imagery, or forming associations between new
information and previously learned information. Hertzog et al. (2008) applied a metacognitive
model of learning about strategy effectiveness in an experiment in which participants learned
paired associates (e.g., table–wallet, apple–book) across successive lists. The model posits that
individuals monitor their performance on the specic task and link their strategies to these
outcomes. Participants learned that interactive mental imagery was an effective strategy for
learning paired associates and that this learning depended on monitoring test performance
following each strategy.Hertzog et al.’s ndings underscore the importance of receiving feedback
across successive tests to enable strategy updating, allowing participants to better distinguish
between more and less effective encoding strategies. These results imply that, in our daily lives,
we update our strategies in situations in which we have repeated experience with remembering.
www.annualreviews.org •Memory and Reward-Based Learning 15.5
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
For example, people may have learned to visualize the groceries they need to buy as they have
learned this helps them remember the items, or they have learned to make semantic associations
between the names and the characteristics of colleagues they want to remember at a conference.
3.1. Self-Regulated Learning
Metacognitive awareness of effective strategies and the ability to apply them are of key importance
in the classroom. The ability to engage in such self-regulated learning has received attention as
important for student success. Self-regulated learning refers to thoughts and actions oriented to
students’ goals (Boekaerts 1999). Metacognitive awareness of learning strategies used, and the abil-
ity to update them on the basis of experience, is critical to effectively regulate one’s own learning.
In the context of the classroom, factors such as the ability to plan when to study, manage resources,
and set reasonable goals are important (Pintrich 1995). Studies of metacognition in strategy updat-
ing are also relevant. For example, feedback in the form of frequent quizzes can enable updating
of study strategies (cf. Soderstrom & Bjork 2014). As in the laboratory, knowledge of effective
study strategies may be tied to specic contexts; for example, a student may develop an effective
strategy of retrieval practice in a foreign language class but may not transfer this knowledge to a
math class (Hadwin et al. 2001; see also McDaniel & Einstein 2020).
Given the learners’ ability to update encoding strategies on the basis of experience, it seems
likely that they can also apply these strategies selectively based on item value. However, this appli-
cation may be calibrated to item value. That is, although learners may be aware that certain deep
encoding strategies are effective, such deep encoding strategies are more effortful, making it coun-
terproductive to apply them indiscriminately to irrelevant and relevant information. The value of
information may therefore be an important cue for engaging in differential encoding strategies
(Cohen et al. 2014, 2017). In a value-directed remembering paradigm in which participants are
given successive lists of items of varying value and recall items to maximize score, participants
learn to selectively encode high-value items across successive lists of items. Thus, as is character-
istic of metacognitive models, updating encoding strategy relies on experience with tests. Here,
the experience results in the participant not applying a more effective strategy overall but rather
applying this effective strategy more selectively to high-value information.
3.2. Neural Mechanisms of Strategic Learning of Valuable Information
Neuroimaging studies support the idea that selectivity for valuable items in memory arises as a
result of engaging brain regions important for deep semantic processing and reward-motivated
remembering in both younger and healthy older adults (see Bowen et al. 2020; Cohen et al. 2014,
2016). Cohen et al. (2014) administered a version of the value-directed remembering task in which
participants studied lists of words of different values in an fMRI scanner. Participants attempted
to recall items after each list was presented and were given their point total for the list after their
recall attempt. The values were given before each word so that activity related to processing value
would be distinguished from activity associated with encoding the word. The key analysis relied
on the fact that participants varied in the extent to which they showed selectivity for value: Some
participants recalled only the highest-value words and others recalled words irrespective of their
value. Cohen et al. (2014) found that in the left inferior frontal gyrus and the posterior middle
temporal gyrus, the magnitude of the difference between high- and low-value words during
encoding correlated with participants’ selectivity for value. That is, in participants who selectively
remembered high-value words, activation of these left-hemisphere regions showed large differ-
ences for high- and low-value words. For those participants who were indifferent to value, there
15.6 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
R L
z
–3.4 3.4
0
Figure 2
Regions in which the blood-oxygen-level-dependent signal difference between high- and low-value items
was correlated with selectivity. Regions include the left inferior frontal gyrus and posterior middle temporal
gyrus. Figure reproduced from Cohen et al. (2014).
was little difference in these regions during encoding of high- and low-value words (Figure 2).
These value-selective regions correspond to a reverse inference map generated by the Neurosynth
database (Yarkoni et al. 2011) that quanties the probability of a voxel being active in studies that
heavily use the term semantic. Thus, this pattern is consistent with the idea that selective par-
ticipants differentially engage semantic processing regions while encoding valuable items. That
participants learn to differentially apply these strategies through experience with successive lists
is consistent with metacognitive models of strategic learning. As we discuss more thoroughly in
Section 6, memory selectivity is preserved in healthy older adults despite reductions in the
amount of information remembered (Castel 2008, Castel et al. 2012). Engaging semantic pro-
cessing regions during the encoding of high-value words was present in older adults who were
selective for value on a value-directed remembering task (Cohen et al. 2016). This nding is
consistent with evidence of intact metacognition regarding effective strategy use in older adults
under some learning conditions (Castel et al. 2015, Hertzog et al. 2010).
Anatomical ndings by way of diffusion tensor imaging also support the role of enhanced se-
mantic processing of words in value-directed remembering. The integrity of the uncinate fascicu-
lus, a ber tract that connects ventral frontal and anterior and medial temporal regions, is associ-
ated with recall of high-value, but not low-value, words in younger adults (Reggente et al. 2018).
Although the integrity of both the left and the right uncinate fasciculus was correlated with recall
of high-value items, this effect was more robust in the left hemisphere. The uncinate fasciculus is
thought to be involved in semantic aspects of language, memory, and reward (Harvey et al. 2013,
Visser et al.2010). In older adults,integrity of a different left-hemisphere tract, the inferior fronto-
occipital fasciculus, was correlated with recall of high-value, but not low-value, words (Hennessee
www.annualreviews.org •Memory and Reward-Based Learning 15.7
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
et al. 2019b). The inferior fronto-occipital fasciculus is a long ber tract that interconnects frontal
and multiple posterior regions of the brain. It has been directly implicated in semantic processing
by neuropsychological studies (Binder & Desai 2011). Although both younger and older adults use
differential semantic processing for value-directed remembering, it may be supported by different
white matter tracts in the two groups given changes in brain morphology that occur with age.
The pivotal role of practice and feedback in learning to apply effective encoding strategies
would seem to suggest that these effects would be seen primarily in studies using multiple short
lists followed by free recall of items on those lists, as this would provide clear feedback to the partic-
ipant about memory success. The role of free recall in enhancing strategic encoding of high-value
items was investigated by Cohen et al. (2017). In this experiment, participants studied multiple
lists of words with different point values and were given free recall tests after either all, some, or
none of the lists. At the end of the set of lists, participants were given a recognition test composed
of all studied items and lures. The key analyses involve the items that were correctly recognized
but had not been recalled. If participants had received free recall tests, even if there were only tests
on a few of the lists, participants were more likely to recognize high-value items than low-value
items, and this greater recognition was manifested by increased recollection and familiarity of
high-value old items. In other words, high-value items had generally increased memory strength.
In contrast, when no recall tests were given, high-value items were recognized more than low-
value items were, but this effect was manifested in recollection only. Thus, participants were not
more likely to say high-value items were more familiar than low-value items, nor were they more
likely to recognize high-value items that were given in a plural form when they had been studied in
a singular form. Thus, value did not generally strengthen item memory, but its effect was limited
to creating highly specic episodic memories. On the basis of this pattern of results, Cohen et al.
(2017) inferred that interspersed free recall tests, and the feedback they provided, were necessary
for participants to engage in selective deep encoding of high-value items, as deep encoding leads
to a strengthening of both recollection and familiarity (Ozubko et al. 2012, Yonelinas 2002). In the
absence of such experience with tests, the benet of value was limited to recollection, consistent
with the idea that automatic effects of value involve hippocampal engagement, as this structure
is specically involved in episodic encoding (Adcock et al. 2006, Diana et al. 2007). These results
point to the role of metacognition and experience with retrieval as important for the use of deep,
more effective encoding strategies for valuable information.
Results consistent with this framework were reported by Elliot et al. (2020a), who measured
encephalography (EEG) responses while participants studied words of different values and then
later during a recognition test for those words. This approach relied on previous work showing
that elaborative semantic encoding is associated with a late positive wave originating from frontal
regions (frontal slow wave) (Fabiani et al. 1990). The frontal slow wave was not associated with
subsequent memory for high-value words; rather, differential encoding of high-value items was
associated with an early parietal-based component (P3) that is associated with automatic attention
(Polich 2007). Some similar results were obtained from both younger and healthy older adults
(L.T. Nguyen et al. 2019, 2020), in which people learned to focus on high-value items for a recall
test. Thus, these ndings suggest that applying enhanced encoding strategies for valuable items
follows from monitoring test performance in that participants here were given a single recognition
test after encoding.
During encoding, one candidate strategy is to selectively rehearse high-value items in short-
term memory. Although this may happen for some items, and possibly on the rst list in the
value-directed remembering task, there is accumulating evidence that both younger and older
adults use deeper, semantic-based processes, such as imagery or creating vivid associations with
high-value words, to remember high-value words. Some of this evidence comes from self-report
15.8 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
as well as neuroimaging studies in which regions of the brain associated with semantic processing
are activated for higher-value words relative to lower-value words (see Figure 2). In addition, on
recognition memory tests, both younger and older adults often have more detailed recollection for
high-value words (Hennessee et al. 2017, 2018; Villaseñor et al. 2021). For some task-irrelevant
details (such as the color in which a word was presented during encoding), both younger and older
adults show poorer memory for these supercial details of the higher-value words, suggesting that
they are focused on the semantic aspects and that the color details are not well encoded (Hennessee
et al. 2018). Taken together, this nding suggests that recollection may accompany memory for
high-value information but that this recollection is specic to the semantic properties of the items
and may in fact trade off with more supercial, task-irrelevant details of the high-value items.
Despite the evidence that test experience is necessary to strategically encode valuable items,
other evidence suggests that, under some circumstances, people may vary their encoding strategy
for high-value items without this experience. Hennessee et al. (2019a) found that when participants
were required to learn a list of items with different values by using the same encoding strategy
for every item, such as an effective one like mental imagery or an ineffective one like repetition,
effects of value were substantially diminished on a recognition test. This nding contrasted with
a nding for a group learning the same list without being told how to encode items. This group
showed robust effects of value, with performance on high-value items similar to that of the group
engaging in deep encoding for all items and performance on the low-value items similar to that of
the group engaging in shallow encoding for all the items. The interpretation of these ndings is
that by requiring all items to be encoded in the same way, participants are not able to adjust their
encoding strategy by item value. That controlling encoding strategy decreased the effects of value
suggests that when participants are left to their own devices, differential encoding of items by value
appears to occur even without test experience. It seems likely that strategic encoding of items by
value might occur without testing feedback if participants had sufcient metacognitive knowledge
of effective encoding strategies, if participants were motivated to maximize performance, if item
values were distinct, or a combination thereof. Monitoring performance outcomes as they relate
to encoding strategies clearly enhances differential application of effective encoding strategies for
valuable items. However, prior knowledge of effective encoding strategies may also be applied in
new situations when people are motivated to remember highly valuable information.
4. AUTOMATIC EFFECTS OF VALUE ON MEMORY
Given the importance of differential learning of information that is valuable, it is not surprising
that neural mechanisms have evolved to enhance encoding of items that are associated with reward
or anticipated reward. If these mechanisms evolved for this purpose, rst, they would be expected
to be conserved across mammalian species and would enable organisms to prioritize storage of in-
formation that may be important for survival. Second, these mechanisms would not require a con-
scious decision on the part of the learner to engage in effortful encoding processes but would occur
automatically when high-value information is encountered. Finally, these mechanisms would in-
volve modulating memory systems by neural circuitry involved in reward processing.
4.1. Interactions Between Reward Circuitry and the Hippocampus
Previous work on memory for value has generally focused on brain reward systems and how
they interact with medial temporal lobe regions that subserve memory, such as the hippocampus
(Shohamy & Adcock 2010). An example of this approach was an inuential neuroimaging study
in which participants studied pictures associated with different amounts of money with the
www.annualreviews.org •Memory and Reward-Based Learning 15.9
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
instruction that they would receive this amount of money for correct subsequent recognition of
the picture (Adcock et al. 2006). As expected, participants recognized more of the high-payoff
pictures at test. The key nding was that, for high-value items, subsequent memory was associated
with greater activation in midbrain dopaminergic regions and enhanced functional connectivity
between these regions and the medial temporal lobe (see also Bowen et al. 2020). These results
suggest that coactivation of midbrain reward regions and medial temporal lobe enhanced encod-
ing. Previous research has identied midbrain regions, particularly the ventral tegmental area
(VTA), that are the source of dopaminergic input to the cerebral cortex and limbic system as
those involved in reward (Fibiger & Phillips 1986). Dopaminergic input from the midbrain is
thought to modulate hippocampal activity during learning and thereby enhance encoding and
hippocampus-dependent consolidation of memory (Gruber et al. 2016).
There is ample anatomical evidence for direct connections between the main hub of the mid-
brain reward system, the VTA, and the hippocampal formation (Gasbarri et al. 1994, 1997; Jay
2003) that support functional connectivity between these structures. Dopamine’s role as a neu-
romodulator that contributes to neural plasticity was the focus of seminal models described by
Lisman and colleagues (Lisman & Grace 2005, Lisman et al. 2011). In the 2011 model, dopamine
is required to stabilize plasticity generated by activation of the N-methyl--aspartate (NMDA)
receptor. Long-term learning depends on a Hebbian mechanism, by which co-occurring inputs
result in synaptic activation in the presence of postsynaptic depolarization. NMDA activation, as
the result of this co-occurrence, leads to synapse-specic strengthening. Dopaminergic inputs are
additionally needed to maintain this increase in synaptic strength. Support for this neo-Hebbian
model of neural plasticity comes from studies such as one in which dopaminergic antagonists do
not prevent learning of episodic place–cue associations when tested after 30 min but impair perfor-
mance after a 1-day delay (Bethus et al. 2010). According to the Lisman et al. (2011) neo-Hebbian
model, all associative information is stored initially by Hebbian processes in the hippocampus,
but only the fraction that is accompanied by a dopaminergic signal is maintained in memory. As
dopaminergic activity signals novelty, salience, and reward (Berridge 2006, Schultz 2007), this
feature ensures preferential encoding of that which is deemed important.
4.2. The Role of Dopamine
The putative role of dopamine in strengthening memory appears to occur independently of intent
to learn. For example, simply presenting an unexpected reward led to better encoding of tempo-
rally proximal stimuli in an incidental memory paradigm (Murayama & Kitagami 2014). This
nding suggests that dopamine release occurring as a consequence of reward led to enhanced
encoding by making hippocampal processing more effective. Potentially rewarding stimuli may
automatically lead to more effective encoding, perhaps via a mechanism laid out by Lisman et al.
(2011). This effect was observed only after a 1-day delay, consistent with the idea that the role of
dopamine is to stabilize rather than drive neural plasticity.A similar nding was obtained by Braun
et al. (2018), who provided rewards as participants traversed a maze on a computer screen. Ob-
ject images and their location in the maze were better remembered on the basis of their temporal
and spatial proximity to the reward. Thus, the reward led to a graded retroactive strengthening
of memory for these arbitrary stimuli. As in the Murayama & Kitagami (2014) study, these effects
were present only after a 24-h delay. It also appeared that having an interval of rest after the re-
ward was obtained potentiated this strengthening, suggesting that neural replay of the preceding
events may be the way that these memories can be strengthened by a reward that occurs later in
time (Braun et al. 2018). These results indicate that the effects of reward on incidental learning
can effectively extend back in time to facilitate encoding of sequences of behavior.
15.10 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
The midbrain dopamine signal has been associated with several processes in reward-related
behavior. A dominant view is that it signals a prediction error when reward is greater or less than
what is expected rather than signaling the presence of reward itself. Dopaminergic neurons are
activated above baseline ring when rewards are higher than predicted and show a decrease in r-
ing below baseline when rewards are lower than expected. Expected rewards do not change ring
(Schultz 2016). By this view, the dopaminergic signal would automatically strengthen memories
only for unexpected rewarding outcomes. For example, if I visit a restaurant that greatly exceeds
my expectations, memory for this experience may be greater than if the restaurant had been highly
recommended. It may be that the strategic, effortful value-directed encoded processes described
above are necessary to enhance encoding of expected valuable information.
The role of dopamine is not only to signal the presence of unexpected reward itself but also to
code the incentive salience of stimuli (Berridge & Robinson 1998). Dopamine activity is reecting
not merely the experiencing of reward but rather the wanting of rewarding stimuli, attributing
value to cues that predict reward, enabling these stimuli to grab attention and inducing motivation
to obtain the reward. This role of dopamine is evidenced by neurons in the VTA that re in
anticipation of rewards, often more than in response to the reward itself (Ferguson et al. 2020,
Kosobud et al. 1994, Ljungberg et al. 1992). Thus, automatic strengthening of memory may occur
for information associated with anticipated rewards, as in a value-directed remembering paradigm
in which the items are paired with values that will be obtained after successful recall or recognition
of the item. One may also nd it relatively easy to memorize the address or phone number of a
person one has a crush on, or of a delicious take-out pizza place, via this mechanism.
Midbrain dopamine is also sensitive to novelty. Wittmann et al. (2007) showed that the
dopaminergic midbrain blood-oxygen-level-dependent signal increased in response to cues pre-
dicting novelty and to unexpected novel stimuli, similar to responses to reward. These results could
be interpreted as novelty being intrinsically rewarding or as evidence that both reward and novelty
are processed by anatomically overlapping circuits. Novelty is thought to engage a hippocampus–
VTA loop in that novelty detection by the hippocampus drives ring of dopaminergic neurons in
the midbrain via its output from the subiculum to the nucleus accumbens and ventral pallidum
(Lisman & Grace 2005). This dopamine signal enhances learning of this novel information by fa-
cilitating hippocampal plasticity. Of note, Wittmann et al. (2007) observed that novelty enhanced
recollection of items on a subsequent recognition test, suggesting that these factors specically
enhance episodic encoding. The hypothesized role of plasticity in the hippocampus in automatic
strengthening of memory through reward or novelty is consistent with a selective role of the hip-
pocampus in episodic memory (Eldridge et al. 2000). Emotional arousal and stress may also play
a key role especially in the context of memory and aging (for reviews, see Bowen 2020, Madan
2017, Mather 2016), leading to selective attention and biases in memory for younger and older
adults.
Studies of the effects of value on memory have typically either emphasized putative dopamine-
driven automatic effects or strategic metacognitive effects in isolation. However, remembering
valuable information likely benets from both mechanisms. There have been several attempts to
disentangle the effects of value per se on memory from the differential processing of valuable
information in which participants are intentionally engaged. One approach has been to present
words with values and then give an unrewarded test of memory for the words (Madan et al. 2017).
Participants chose one of two words in each trial and earned either a high or a low reward for
the choice. Participants had better recall for words that had earned a high reward during the
choice task even when there was no incentive to preferentially remember them. Another approach
has been to examine the effect of value on memory independently from the effects of value on
www.annualreviews.org •Memory and Reward-Based Learning 15.11
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
usefulness (Chakravarty et al. 2019). In this study, participants earned points by remembering to
choose a high-value word and remembering to avoid choosing a low-value word, so both types
of words were equally useful to encode and it benetted participants equally to engage in deep
encoding of both types of items. This study showed a small effect of value on output order in free
recall independent of whether the item was useful to remember (see also Madan et al. 2012). One
interpretation of these ndings is that they may reect the automatic strengthening of memory
by value that increased the item’s accessibility independent of the participant’s intent to learn the
item.
Another approach has been to use a directed forgetting procedure (Gardiner et al. 1994), in
which items at study are designated either to-be-remembered [learn cue (L)] or to-be-forgotten
[forget cue (F)]. Participants are actually later tested on both L- and F-cued items. From the par-
ticipant’s point of view, engaging in deeper semantic processing,such as reecting on the meaning
of the word, vividly imagining its referent, or associating the word with similar words presented
earlier,is wasted effort if the word is designated to be forgotten (Bjork & Woodward 1973,Murphy
& Castel 2021, Popov et al. 2019). Thus, effects of value that persist for to-be-forgotten items are
likely to reect automatic strengthening of these memories.
Using a directed forgetting approach, Hennessee et al. (2019a) presented both high- and low-
value words that were designated to be either learned or forgotten. On a subsequent recognition
test, all studied words were presented along with new words. As shown in Figure 3, participants
better recognized items that they were cued to learn than items they were cued to forget, demon-
strating that participants were differentially encoding on the basis of the instructional cues. Partic-
ipants also better recognized the high-value words than the low-value words. The most interesting
nding was the substantial effect of value on items that participants were told to forget. These data
indicate that even when participants were encouraged not to effortfully encode items, high-value
items were nonetheless encoded more effectively.Hennessee et al. (2019a) suggested that this en-
hancement is nonstrategic and arises automatically through the engagement of midbrain reward
circuitry. This enhancement was seen after a brief delay, thus implying that automatic effects of
value may not require a long delay to appear.
0.50
0.55
0.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
1.00
Forget Remember
Proportion hits
Low value
High value
Figure 3
The proportion of hits for high- and low-value words is shown as a function of the forget or remember cue
in the directed forgetting task. High-value items were later remembered better than low-value items when
participants were told to forget them immediately after presentation of the item (data replotted from
Hennessee et al. 2019a).
15.12 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
5. THE DEVELOPMENT OF VALUE-DIRECTED REMEMBERING
Childhood is a time of concentrated learning of skills and facts about the world. While infants
and young children acquire motor skills and language at a rapid rate, episodic memory abilities
continue to improve throughout the elementary school years (Ghetti & Bunge 2012). This abil-
ity to remember events, and thus bind the elements of the episode to its specic spatiotemporal
context, depends on the medial temporal lobe, particularly the hippocampus (Davachi et al. 2003,
Eldridge et al. 2000). In addition, effective episodic memory depends on the ability to strategically
encode information, temporally order information, and monitor the contents of retrieval. These
executive functions depend on lateral prefrontal cortex, with other cerebral cortical regions also
playing a role (Blumenfeld & Ranganath 2007). The hippocampus matures within the rst three
years of life yet changes in microstructure as a result of synaptic pruning continue to occur into
adulthood (Gogtay et al. 2006). Development of prefrontal cortex is even more protracted, with
substantial changes in cortical thickness occurring through adolescence (Giedd 2004, Ofen et al.
2007). Increased engagement of prefrontal cortex in episodic memory tasks occurs from childhood
through adolescence, correlated with performance (Wendelken et al. 2011). Importantly, commu-
nication between the medial temporal lobe and prefrontal cortex likely increases from childhood
into adolescence, supported by changes in white matter connectivity between these regions (Lebel
& Beaulieu 2011).
Much of the developmental trajectory of episodic memory reects the changes that occur
throughout childhood and adolescence in regions involved in executive control processes. Older
children may be able to engage in more semantic encoding strategies that require cognitive control
and to use retrieval strategies more effectively. Similarly, value-directed remembering continues
to improve until adulthood. Castel et al. (2011a) examined value-directed remembering in indi-
viduals between 5 and 96 years of age to compare differences between the development of verbal
memory ability and the development of selectivity for value (Figure 4). A selectivity index (SI)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
ADHD Alzheimer’s
disease
Proportion of words recalled
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Selectivity index
Age group
Children Adolescents Younger
adults
Middle-aged
adults
Younger old
adults
Older old
adults
Figure 4
The proportion of words recalled (plotted in blue) in the selectivity task for different age groups across the
life span, and the selectivity index (plotted in red) for each age group, including groups with attention decit
hyperactivity disorder (ADHD) (stars) and early-stage Alzheimer’s disease (circles) (data replotted from Castel
et al. 2009, 2011a,b).
www.annualreviews.org •Memory and Reward-Based Learning 15.13
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
has been utilized in prior work (Castel et al. 2002, 2007; Hanten et al. 2007) that measures how
sensitive people are to remembering higher- versus lower-value words. This index is based on the
participant’s score (the sum of the points that were paired with the recalled items, or the value of
the recalled items) relative to chance and ideal performance. For example, if a given participant
remembered four words, and the points associated with the words were 12, 10, 9, and 8, that par-
ticipant’s SI would be considered high. The ideal score for four words is 12 +11 +10 +9=42,
whereas the score of the participant in question is 39. A chance score is based on calculating the
average value of the points (using a 12-word list, with numbers ranging from 1 to 12, the average
would be 6.5) and multiplying that value by the number of words recalled (in this case, four). Thus,
theSIinthiscaseis(39−26)/(42 −26) =0.81. Note that the index can range from +1to−1.
Perfect selectivity would result in an SI of 1.0, whereas selection of words with the lowest values
(e.g., recalling the 1-, 2-,and 3-point words) would result in an SI of −1.0. A set of words recalled
with no regard to their point values (i.e., showing no selectivity) would result in an SI close to 0.
Thus, the SI provides a selectivity, or efciency, index based on one’s actual score, relative to an
ideal score, accounting for the number of words recalled. The age groups that were tested differed
in terms of number of words recalled per list, with a sharp increase from childhood to adolescence,
and a smaller further increase from adolescence to young adulthood. Not surprisingly, the num-
ber of words recalled dropped substantially in middle age, further declining with advancing age.
When the degree to which valuable items were selectively remembered was assessed, a different
pattern emerged. Both children and adolescents had substantially lower levels of selectivity than
did younger adults. However, unlike total number of words recalled, selectivity did not decline
until participants were in the oldest age group (mean age: 84 years), and even this group showed
greater selectivity than children and adolescents. These results suggest that the ability to selec-
tively apply encoding strategies to high-value items is not fully apparent until adulthood. This
selectivity relies on metacognitive insights into memory capacity, effective encoding strategies,
and the ability to adjust on the basis of feedback. It may be that extensive experience with how
one’s memory works, likely achieved through secondary schooling, is important to develop this
metacognitive awareness.
In this study, participants studied successive lists and recalled words from them. Thus, it is likely
that participants relied on the strategic differential encoding of high-value items. It is possible
that the use of a recognition task that is more sensitive to automatic effects of value would reveal
stronger effects of value, particularly in adolescence, as this stage of life is associated with greater
reward sensitivity (Galván et al. 2006). Although it is tempting to view the gradual rise in value-
directed remembering into adulthood as a function of frontal lobe development, it appears to be
the case that this ability is not susceptible to decline in frontal lobe function that may accompany
aging (Zanto & Gazzaley 2019). It may be that older adults can rely on their experience with
the constraints of their memory and on their knowledge of semantic strategies to maintain the
ability to selectively remember valuable information. Another important nding from Castel et al.
(2011a) was that, despite decits relative to those of adults, even young children were able to show
signicant sensitivity to value in this task. Thus, the ability to prioritize information in memory is
present to some degree in school-age children. Selectivity is, however, impaired in children with
attention decit hyperactivity disorder,although these children recalled a similar number of words
as did children in the control group (Castel et al. 2011b). This particular decit in cognitive control
may have functional consequences in school settings, where some information is prioritized over
other information for learning.
As children grow older,and the demands of school become more intense, the ability to manage
one’s own learning becomes increasingly important. This is particularly true for students embark-
ing on higher education. College students are often used as a classic example of the importance
15.14 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
of value-directed remembering. Courses often present an overwhelming amount of information
to the learner, and the successful student must prioritize information that is more valuable or
more important to study in order to perform well. Research on self-regulated learning focuses on
metacognitive factors, which may play a role in student success (Isaacson & Fujita 2006). There
are also substantial individual differences in the ability to selectively remember valuable informa-
tion (Elliot et al. 2020b), and these differences may play a role in students’ success independent
of scholastic aptitude. Focus on improving the ability to prioritize important information may be
most crucial in the sciences, where a substantial amount of content knowledge must be learned
to proceed to more advanced study. The ability to selectively prioritize encoding of high-value
information does not correlate much with working memory capacity (Grifn et al. 2019, Robison
& Unsworth 2017). Although working memory is highly associated with many higher cognitive
functions, including recall (Unsworth 2010, 2016), it is encouraging that people of a range of
cognitive abilities appear to have the capacity to improve functional memory by learning how to
selectivity remember valuable items.
6. MOTIVATED MEMORY AND AGING: AN ENHANCED
FOCUS ON VALUE?
As people get older, they tend to experience a variety of memory changes and cognitive impair-
ments (Salthouse 2019, Thomas & Gutchess 2020). Older adults may be especially distressed by
memory changes and challenges (Kinzer & Suhr 2016). Memory changes, such as forgetting the
name of a recent acquaintance or where one parked the car, can be frustrating. Although people
of all ages experience memory challenges, older adults may be more concerned and worried than
younger adults that these changes could signal early stages of dementia, leading to greater anxiety
about memory failures (Mazerolle et al. 2017).
However, despite the memory challenges and impairments that older adults experience, healthy
older adults do show sparing of certain memory functions, such as semantic knowledge, use of a
sense of familiarity,and other forms of memory that can help compensate for memory loss (Garrett
et al. 2010, Nyberg & Pudas 2019). Specically, older adults may become more adept at using
memory strategies to offset forms of memory loss (Hertzog & Dunlosky 1996). These strategies
include making lists, creating reminders, and putting things in familiar places for future use, such
as keeping keys in the same place each day so that one does not need to search for them.
Another strategic approach that older adults utilize is to selectively focus on remembering im-
portant things or events, sometimes at the expense of forgetting or not paying attention to less
important information (Castel 2008, Castel et al. 2012, Hargis et al. 2019). This selectivity pro-
cess can help older adults focus on and remember what matters most (such as positive emotional
events or important information to use in the future) and may be a potent mechanism that allows
older adults to function effectively in numerous settings. Although older adults display memory
decits, a large variety of research ndings support the notion that older adults can remember
high-value/more important information and that age-related differences are much more apparent
for low-value/less important information (see Figure 1b). Thus, there may be an age-related im-
pairment in memory capacity but a sparing (and sometimes enhancement) in the use of selectivity
or prioritizing high-value information in memory.One recent perspective (see Figure 5) suggests
that cognitive mechanisms may be impaired in older age but that more motivational and strate-
gic processes can then compensate for these cognitive impairments (Swirsky & Spaniol 2019). At
the simplest level, this can be found in a laboratory-based, value-directed remembering selectiv-
ity task (see Figure 1) in which older adults can engage in selective memory by remembering
the high-value items. This can often result in memory performance similar to that of younger
www.annualreviews.org •Memory and Reward-Based Learning 15.15
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
Sources of decline in cognitive
selectivity with aging:
• Cognitive control
• Inhibition
• Arousal
Sources of stability/increase in
motivational selectivity with aging:
• Emotion regulation
• Intrinsic motivation
• Reward sensitivity
Behavioral age eects:
• ↓ Selective attention
• ↑ RT to target in visual search
• ↓ Suppression of attraction
• Hyperbinding and associative
decit (↓ memory selectivity)
Candidate neural substrates:
• ↓ PFC structure and function
• ↓ FPCN modulation by DA and NE
• De-coupling of DMN and FPCN
Age-related decline
Age-related
preservation/increase
Increase relative to younger adults
Decrease relative to younger adults
Stable relative to younger adults
↑
↓
=
Cognitive
selectivity
Motivational
selectivity
Behavioral age eects:
• ↑ Positivity eect
• ↑ Selective engagement in
meaningful tasks
• = Reward-modulated attention
and memory
Candidate neural substrates:
• = Amygdala, ventral striatum
structure and function
• ↑ FPCN modulation (DA-driven?)
Figure 5
A summary of age-related differences (both decline and preservation/increase) in terms of cognitive and
motivation selectivity. Declines in processes including cognitive control and inhibition can be compensated
for by increases in emotional control and motivation. Putative neural substrates are also shown. Figure
adapted from Swirsky & Spaniol (2019). Abbreviations: DA, dopamine; DMN, default mode network;
FPCN, frontoparietal control network; NE, norepinephrine; PFC, prefrontal cortex; RT, radiotherapy.
adults for these high-value words, after some task experience, in which older adults become aware
of how many items they can selectively remember and then focus on this amount, suggesting a
metacognitive component is involved.
A great deal of the value-directed remembering work has used words paired with point val-
ues to engage reward-based memory,but the value-directed remembering paradigm has also been
extended to examine other more real-world challenges relevant to older adults, such as remem-
bering important information about medication (Friedman et al. 2015, Hargis & Castel 2018),
gains and losses in a nancial context (Castel et al. 2016), and important social information about
people you recently met (Hargis & Castel 2017). In addition, older adults can retain high-value
information for longer delays, in some cases showing recognition (picture memory) benets that
were not present on immediate test but were present at a 24-h delay (Spaniol et al. 2014). This
nding suggests that reward-enhanced memory may be driven by consolidation, as similar long-
term memory benets for high-value information have been found in younger adults following
sleep (Lo et al. 2016).
The ability to be selective likely relies on brain mechanisms that develop and also decline over
the life span, perhaps reecting frontal lobe functions such as cognitive control. Reward-based
memory and value-directed remembering may also be useful to detect decits in attention and
memory that may be related to the onset of dementia. In the early stages of Alzheimer’s disease,
dissociations can also be found between memory capacity and the ability to be selective. People
15.16 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
with early Alzheimer’s disease show more pronounced decits in selectivity despite still being able
to recall both higher- and lower-value information,suggesting that the ability to selectively attend
to high-value items, at the expense of lower-value information,may be impaired in early stages of
dementia (see Castel et al. 2009, Wong et al. 2019). This decit in prioritizing appears to be most
pronounced in older adults with behavioral-variant frontotemporal dementia (FTD) relative to
those with Alzheimer’s disease (Wong et al. 2019), again implicating the frontal lobes as critical
for maintaining selectivity and engaging/regulating reward-based memory.
7. METACOGNITION GUIDING VALUE-DIRECTED REMEMBERING
When we are faced with large amounts of information, we often need to select what is most im-
portant to remember. Being aware that memory capacity is limited, and may be more so in older
age, suggests that this metacognitive insight can help people focus on what is most important.
Specically,if one knows they cannot remember large amounts of information, they may be more
inclined to selectively focus on a smaller amount of information that is highly relevant to future
goals. For example, if you are going on an international trip and are rushing to pack, you may
rst make sure you remember highly important things, such as your passport, phone, and keys,
and be less concerned if you forget other items, such as a toothbrush or magazine (McGillivray
& Castel 2017). Thus, older adults may be more responsible about remembering what is most
critical (Murphy & Castel 2020, 2021), perhaps at the expense of less relevant things, on the basis
of schemas and prior knowledge developed over a lifetime. In addition, when rushing, people may
forget things, but some research shows that rushing can also lead to spared selectivity, such that
people focus on higher-value items at the expense of lower-value items (Middlebrooks et al.2016).
In some cases, having schemas or experience with certain scenarios can help older adults focus on
remembering what is important, such as when to take medications, and on using these established
knowledge structures to facilitate remembering what is essential.
Evidence regarding the metacognitive aspects of reward-based memory and value-directed
remembering comes from asking people to predict which items they will remember. Often, people
initially tend to think they will remember more than they actually do, although this overcondence
is reduced with some task experience (McGillivray & Castel 2011, Siegel & Castel 2019, Siegel
et al. 2020). To encourage a good match between what people say they will remember and what
they actually later recall, researchers have developed a betting procedure, such that if a participant
bets on a word–value pair (e.g., table–9) and later recalls “table,” they then receive the 9 points,
but if they bet on the word and fail to recall it, then they lose the 9 points (McGillivray & Castel
2011). On the initial list, both younger and older adults tend to bet that they will remember more
words than they actually do, sometimes resulting in a negative or near-zero score on the recall
test (see Figure 6). However, on subsequent lists (with new word–value pairings) participants
become more metacognitively aware, betting on fewer words and also more likely to recall the
higher-value words that they bet on; this is especially the case for older adults. In fact, by the last
few lists, younger and older adults achieve a similar score, as the older adults recall fewer words
but have learned to bet more accurately on the high-value words that are recalled, demonstrating
calibration and metacognitive awareness regarding selectivity (Murphy et al. 2021, Siegel & Castel
2019). This may also illustrate a form of responsible remembering in which the older adults are
most likely to remember the words they said they would remember and forget the ones that they
did not bet on remembering (Murphy & Castel 2020). However, stress and stereotype threat may
disrupt this metacognitive process in older adults (Fourquet et al. 2020), suggesting anxiety about
memory can impact performance in older age in a variety of settings (Mazerolle et al. 2017).
www.annualreviews.org •Memory and Reward-Based Learning 15.17
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
–10
0
10
20
30
40
50
123456
Average score
List
Young
Old
Figure 6
The average score obtained by younger and older adults when betting on which words will be remembered
in the selectivity task. Although both age groups bet on more words than they recall on earlier lists, older
adults reach a score similar to that of younger adults on later lists by recalling fewer words but are more
accurate in terms of betting on the words that are recalled (data replotted from McGillivray & Castel 2011).
Self-regulated learning strategies in value-directed remembering, in which participants can
choose which and how long to study each item-value pair, likely play an important role in reward-
based memory. Specically, participants can engage in more selective processing when they are
presented items simultaneously (i.e., participants can engage self-regulated learning) as opposed to
sequentially (i.e., participants have less control) (Middlebrooks & Castel 2018). This strategy may
be especially important for older adults in order to compensate for age-related sensory slowing and
memory impairments (Castel et al. 2013). In a modied value-directed remembering self-paced
study paradigm, participants are simultaneously shown values ranging from 1 to 30 and can choose
which value to study (by clicking on the value on the screen with a mouse to reveal the associated
word). After a 2-min encoding session, in which participants can study any of the items for any
length of time and also revisit items, participants then are given a test on which they are asked to
recall the words in order to maximize their score. Following this test, participants engage in several
more lists to determine how task experience modies how people choose which items to study and
how long to study each item. Under these self-regulated encoding conditions, older adults tend to
spend disproportionately more time studying the higher-value items than younger adults do, but
both younger and older adults choose to revisit the higher-value items more frequently (Castel
et al. 2013, Li et al. 2018, Middlebrooks & Castel 2018).
Although there is some memory capacity component to value-directed remembering that is
likely inuenced by working memory abilities and individual differences (Grifn et al. 2019, Hayes
et al. 2013), working memory may not always be strongly related to the ability to be selective,
likely because of the metacognitive strategies that can be implemented (selectively focusing on the
high-value items). As such, some research has shown individual differences that can be related to
working memory,whereas other work has found small or no reliable correlations in either younger
or older adults (Castel et al. 2009, Cohen et al. 2014, Middlebrooks et al. 2017). In addition, the use
of pupil dilation as a measure for attention and reward-based encoding indicates high-value items
elicit greater pupillary dilation than do lower-value items in younger adults (Ariel & Castel 2014),
but it is unclear whether this also occurs in older adults, who may be using effective metacognitive
strategies to offset any potential age-related decits in the more automatic release of dopamine
that can inuence reward-based memory.
15.18 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
–80
–60
–40
–20
0
20
40
60
80
–100 –10 –1 1 10 100
Estimated amount borrowed or owed ($)
Actual amount borrowed or owed ($)
–$100
+$100
Young
Old
Figure 7
The average estimated amount of borrowed or owed money that younger and older adults remembered
when earlier presented with faces paired with values that indicated the amounts borrowed from or owed to
the participant. The results show that older adults tend to focus more on remembering the gains and to
underestimate losses (data replotted from Castel et al. 2016).
8. BINDING AND SPATIAL MEMORY
Reward-based memory is often studied by pairing words or images with some form of monetary
incentive (Adcock et al. 2006, Spaniol et al. 2015) or point values (Castel 2008), but other domains
engage these value-driven memory processes. For example, you may meet someone who is impor-
tant and need to remember their name for a future interaction (e.g., a doctor, a new friend). In one
study, participants viewed faces paired with a person’s name, profession, and the likelihood that
they would see this person in the future. Both younger and older adults selectively remembered
the people and professions that they felt were important (Hargis & Castel 2017), suggesting that
this subjective form of importance guides value-directed remembering. In another study, faces
were paired with dollar amounts that reected how much the face owed the participant or how
much the participant owed the face, representing potential gains and losses. The magnitude of
the dollar amount inuenced the later recall of these amounts, but older adults tended to better
remember the gains than the losses and tended to underestimate memory for the loss amount,
whereas younger adults did not show this bias (Castel et al. 2016) (see Figure 7). In addition, both
younger and older adults may feel that if information is initially forgotten then it is not valuable
(Witherby et al. 2019). Thus, there may be subjective age-based biases in how younger and older
adults pay attention to what is deemed important.
Effects of reward-based remembering on binding objects to spatial locations have also been
investigated. As practical examples, we are often challenged to remember where we put our keys
or the location of a restaurant we visited a few months ago. Siegel & Castel (2018) developed
spatial selectivity tasks to investigate these issues. In these tasks, participants study a spatial grid
on which different objects appear at different spatial locations and each object is also assigned a
point value. Participants are asked to remember the objects so that they will later have to recall
where on the grid each object is and will be rewarded with the point value assigned to each object
www.annualreviews.org •Memory and Reward-Based Learning 15.19
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
(which is presented at encoding next to the object). At test, participants are shown each object
and must indicate where on the grid the object was. Again, though associative memory decits
in older adults have been widely documented (Ariel et al. 2015, Naveh-Benjamin 2000), older
adults can selectively remember the locations of the high-value objects (Allen et al. 2021, Siegel &
Castel 2018), suggesting that engaging in value-directed remembering extended to the binding of
objects to spatial information can overcome associative memory decits typically found in older
age. In addition, older adults may be biased to remember the location of positive-gain locations (as
opposed to locations where points could be lost; Schwartz et al. 2020), suggesting that selectivity
biases attention in older age toward what is considered to be good.
9. CURIOSITY AND REWARD-BASED LEARNING
One mechanism that guides what people deem important is initial level of interest in a topic area,
or curiosity in learning the answers to questions of interest (Hidi & Renninger 2019). Older adults
may sometimes show a reduction in general curiosity at the trait level (Chu et al. 2020, Sakaki et al.
2018), but they may also show age-related enhancements in learning specic new information,
skills, or trivia that is of use or interest, again suggesting a selective engagement (Hargis et al.
2020, Hess et al. 2018, McGillivray et al. 2015). For example, for each of the following trivia
questions, please rate how curious you are to learn the answer:
What was the rst nation to give women the right to vote?
What was the rst product to have a bar code?
What was Dr. Frankenstein’s rst name?
With what product did the term “brand name” originate?
What is the slowest swimming sh in the world?
People do not know the answer to most of these questions, such that some then elicit a level of
curiosity (and if you are curious, the answers appear in Section 12). In one study, younger and older
adults were shown a normed set of 60 trivia questions and, after each question, were asked how
interested they were in learning the answer, were then asked to guess it, and were then told the
answer (McGillivray et al. 2015). There were large individual differences in the level of curiosity
among those learning the answers to these questions (Fastrich et al. 2018). When people were
given a surprise test for the answers 1 week later, unlike most tests of memory, there were no age
differences in overall memory for the recall of answers to the questions. In addition, older adults
tended to remember the answers to questions they were most interested in, whereas younger adults
did not show this correlational relationship between interest and later memory.Other research has
shown that older adults can benet from initial curiosity in terms of binding related information
(Galli et al. 2018). These ndings t with selective engagement theory (Hess et al. 2018), which
suggests that older adults report higher levels of engagement and demonstrate better memory
performance when intrinsically motivated. Thus, curiosity may lead to a form of reward-based
learning in which people, especially older adults, tend to remember what is most interesting, and
this may be subserved by both automatic and strategic processes that can enhance memory.
Curiosity can lead one to learn information and being curious can enhance learning well
beyond the classroom. Having some level of curiosity in selective domains may be critical as we
get older. Lifelong learning may engage older adults in new activities that they are interested
in learning more about or even mastering (e.g., drawing, music composition, photography,
Spanish). For example, one’s level of cognitive interest was of the strongest forms of motivation
to learn among retired older adults (Kim & Merriam 2004). This curiosity-based engagement
in new skills can also lead to enhancements in other domains of cognition and memory over a
15.20 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
monthslong training period (Leanos et al. 2020), which could also lead to maintaining or even
gains in functional independence in older age (C. Nguyen et al. 2020).
10. FUTURE DIRECTIONS
In this review,we outline a dual-process approach of how strategic and more automatic processes
contribute to reward-based memory.However, many important questions remain unanswered and
we outline a few future directions that may provide a more complete picture. First, it is important
to use experimental procedures that can selectively engage strategic and automatic processes in
reward-based memory. For example, the use of procedures such as transcranial direct current
stimulation that could selectively target strategic processes could provide insight into the specic
contribution of processes to reward-based memory. Second, examining individual differences in
the ability to recruit these different processes would provide useful insight into how training could
enhance selective memory in a variety of contexts, including classroom learning. Third, potential
biases in memory may be the result of a dual-process, reward-based learning approach, such that
people may misremember certain events owing to the relative importance assigned to certain
aspects of a memory. This approach could lead to biases that could make people (and perhaps
especially older adults) more susceptible to gain-based decision-making processes that could result
in being taken advantage of via scams that focus on rewards. Finally, further work is needed to
understand how the nature and timing of rewards can enhance memory in healthy individuals
as well as in individuals with conditions such as schizophrenia and different forms of dementia.
Overall, these directions could lead to important discoveries regarding how, when, and why we
pay attention to certain cues and information in our environment, and provide predictive models
of what will later be remembered on the basis of the relative importance of the information in
question.
11. SUMMARY AND CONCLUSIONS
We live in a world where we are overloaded with information, such that selecting important infor-
mation to remember is critical. In this review, we outline both strategic and more automatic effects
that can lead to memory for both the mundane and the essential. Brain regions that support auto-
matic and strategic effects can be dissociated, and more research is needed to better understand the
behavioral mechanisms and neural substrates that give rise to reward-based memory and learn-
ing. There are clear developmental changes that bring these processes online that have important
implications for both self-regulated and classroom learning. In older adults, despite declines in
memory capacity, healthy aging can lead to a selective focus on important information. When we
are short on time, our metacognition and awareness that we cannot remember everything (or most
things) can guide the use of strategies to digest information and retain what is most important for
the future. Emotion and stress can affect when and how people attend to and remember important
information, and future research must address how effective interventions and training can help
people learn to remember what is most important. For example, if training to go to outer space,
an astronaut must be able to access the most relevant and important information at a specic time.
Accessing and sifting through a large amount of information take time and effort, so one’s exper-
tise and training should lead to a focus on selectivity and not simply enhancing the sheer quantity
of information committed to memory. In general, while most people seek to improve how much
they can remember, a more practical goal, at almost any stage in development, may be to enhance
selectivity to remember what matters most and learn to forget unnecessary information.
www.annualreviews.org •Memory and Reward-Based Learning 15.21
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
12. APPENDIX
The answers to the trivia questions are as follows:
New Zealand
Wrigley’s chewing gum
Victor
Whiskey
Seahorse
DISCLOSURE STATEMENT
The authors are not aware of any afliations, memberships, funding, or nancial holdings that
might be perceived as affecting the objectivity of this review.
ACKNOWLEDGMENTS
The authors thank Tara Patterson, Mary Whatley, Dillon Murphy, Julia Schorn, and Matt Rhodes
for helpful comments.
LITERATURE CITED
Adcock RA, Thangavel A, Whiteld-Gabrieli S, Knutson B, Gabrieli JDE. 2006. Reward motivated learning:
Mesolimbic activation precedes memory formation. Neuron 50:507–17
Allen RJ, Atkinson AL, Nicholls LAB. 2021. Strategic prioritisation enhances young and older adults’ visual
feature binding in working memory. Q. J. Ex p. Psychol. 74(2):363–76
Ariel R, Castel AD. 2014.Eyes wide open: enhanced pupil dilation when selectively studying important infor-
mation. Exp. Brain Res. 232:337–44
Ariel R, Dunlosky J, Bailey H.2009. Agenda-basedregulation of study-time allocation: when agendas override
item-based monitoring. J. Exp. Psychol. Gen. 138:432–47
Ariel R, Price J, Hertzog C. 2015. Age-related associative memory decits in value-based remembering: the
contribution of agenda-based regulation and strategy use. Psychol. Aging 30:795–808
Berridge KC. 2006. The debate over dopamine’s role in reward: the case for incentive salience.
Psychopharmacology 191:391–431
Berridge KC, Robinson TE. 1998. What is the role of dopamine in reward: hedonic impact, reward learning,
or incentive salience? Brain Res. Rev. 28:309–69
Bethus I, Tse D, Morris RGM. 2010. Dopamine and memory: modulation of the persistence of memory for
novel hippocampal NMDA receptor-dependent paired associates. J. Neurosci. 30:1610–18
Bijleveld E, Custers R, Aarts H. 2012. Adaptive reward pursuit: how effort requirements affect unconscious
reward responses and conscious reward decisions. J. Exp. Psychol. Gen. 141:728–42
Binder JR, Desai RH. 2011. The neurobiology of semantic memory. Trends Cogn. Sci. 15:527–36
Bjork RA, Woodward AE. 1973. Directed forgetting of individual words in free recall. J. Exp. Psychol. 99:22–27
Blumenfeld RS, Ranganath C. 2007.Prefrontal cortex and long-term memory encoding: an integrative review
of ndings from neuropsychology and neuroimaging. Neuroscientist 13:280–91
Boekaerts M. 1999. Self-regulated learning: Where are we today? Int. J. Educ. Res. 31:445–57
Bowen HJ. 2020. Examining memory in the context of emotion and motivation. Curr. Behav. Neurosci. Rep.
7:193–202
Bowen HJ, Ford JH, Grady CL, Spaniol J. 2020. Frontostriatal functional connectivity supports reward-
enhanced memory in older adults. Neurobiol. Aging 90:1–12
Braun EK, Wimmer GE, Shohamy D. 2018. Retroactive and graded prioritization of memory by reward. Nat.
Commun. 9:4886
Carr VA, Castel AD, Knowlton BJ. 2015. Age-related differences in memory after attending to distinctiveness
or similarity during learning. Aging Neuropsychol. Cogn. 22:155–69
15.22 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
Castel AD. 2008. The adaptive and strategic use of memory by older adults: evaluative processing and value-
directed remembering. In The Psychology of Learning and Motivation, ed. AS Benjamin, BH Ross, 48:225–
70. London: Academic
Castel AD, Balota DA, McCabe DP. 2009. Memory efciency and the strategic control of attention at encod-
ing: impairments of value-directed remembering in Alzheimer’s disease. Neuropsychology 23:297–306
Castel AD, Benjamin AS, Craik FIM, Watkins MJ. 2002. The effects of aging on selectivity and control in
short-term recall. Mem. Cogn. 30:1078–85
Castel AD, Farb N, Craik FIM. 2007. Memory for general and specic value information in younger and older
adults: measuring the limits of strategic control. Mem. Cogn. 35:689–700
Castel AD, Friedman MC, McGillivray S, Flores CC, Murayama K, et al. 2016. I owe you: age-related similar-
ities and differences in associative memory for gains and losses. Aging Neuropsychol. Cogn. 23(5):549–65
Castel AD, Humphreys KL, Lee SS, Galván A, Balota DA, McCabe DP. 2011a. The development of memory
efciency and value-directed remembering across the lifespan: a cross-sectional study of memory and
selectivity. Dev. Psychol. 47:1553–64
Castel AD, Lee SS, Humphreys KL, Moore AN. 2011b. Memory capacity, selective control, and value-
directed remembering in children with and without attention-decit/hyperactivity disorder (ADHD).
Neuropsychology 25:15–24
Castel AD, McGillivray S, Friedman MC. 2012. Metamemory and memory efciency in older adults: learning
about the benets of priority processing and value-directed remembering. In Memory and Aging: Current
Issues and Future Directions, ed. M Naveh-Benjamin, N Ohta, pp. 245–70. New York: Psychology Press
Castel AD, Middlebrooks CD, McGillivray S. 2015. Monitoring memory in old age: impaired, spared, and
aware. In The Oxford Handbook of Metamemory, ed. J Dunlosky, S Tauber, pp. 519–35. Oxford, UK: Oxford
Univ. Press
Castel AD, Murayama K, Friedman MC, McGillivray S, Link I. 2013. Selecting valuable information to re-
member: age-related differences and similarities in self-regulated learning. Psychol. Aging 28:232–42
Chakravarty S, Fujiwara E, Madan CR, Tomlinson SE, Ober I, Caplan JB. 2019. Value bias of verbal memory.
J. Mem. Lang. 107:25–39
Chu L, Tsai JL, Fung HH. 2020. Association between age and intellectual curiosity: the mediating roles of
future time perspective and importance of curiosity. Eur. J. Ageing 18(1):45–53
Cohen MS, Rissman J, Hovhannisyan M,Castel AD,Knowlton BJ. 2017. Free recall test experience potentiates
strategy-driven effects of value on memory. J. Exp. Psychol. Learn. Mem.Cogn. 43:1581–601
Cohen MS, Rissman J, Suthana NA, Castel AD, Knowlton BJ. 2014. Value-based modulation of memory
encoding involves strategic engagement of fronto-temporal semantic processing regions. Cogn. Affect.
Behav. Neurosci. 14:578–92
Cohen MS, Rissman J, Suthana NA, Castel AD, Knowlton BJ. 2016. Effects of aging on value-directed mod-
ulation of semantic network activity during verbal learning. Neuroimage 125:1046–62
Davachi L, Mitchell JP, Wagner AD. 2003. Multiple routes to memory: Distinct medial temporal lobe pro-
cesses build item and source memories.PNAS100:2157–62
den Ouden HEM, Kok P, de Lange FP. 2012. How prediction errors shape perception, attention, and moti-
vation. Front. Psychol. 3:548
Diana RA, Yonelinas AP, Ranganath C. 2007. Imaging recollection and familiarity in the medial temporal lobe:
a three-component model. Trends Cogn. Sci. 11:379–86
Dunlosky J. 1998. Linking metacognitive theory to education. In Metacognition in Educational Theory and Prac-
tice, ed. DJ Hacker, J Dunlosky, AC Grasser, pp. 367–81. Mahwah, NJ: Lawrence Erlbaum Assoc.
Eldridge LL, Knowlton BJ, Furmanski CS, Bookheimer SY, Engel SA. 2000. Remembering episodes: a selec-
tive role for the hippocampus during retrieval. Nat. Neurosci. 3:1149–52
Elliot BL, Blais C, McClure SM, Brewer GA. 2020a. Neural correlates underlying the effect of reward value
on recognition memory. Neuroimage 206:116296
Elliot BL, Brewer GA. 2019. Divided attention selectively impairs value-directed encoding. Collabra Psychol.
5:4
Elliot BL, McClure SM, Brewer GA. 2020b. Individual differences in value-directed remembering. Cognition
201:104275
www.annualreviews.org •Memory and Reward-Based Learning 15.23
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
Fabiani M, Karis D, Donchin E.1990. Effects of mnemonic strategymanipulation in a Von Restorff paradigm.
Electroencephalogr. Clin. Neurophysiol. 75:22–35
Fastrich GM, Kerr T, Castel AD, Murayama K. 2018. The role of interest in memory for trivia questions: an
investigation with a large-scale database. Motiv. Sci. 4:227–50
Ferguson LM, Ahrens AM, Longyear LG, Aldridge JW. 2020. Neurons of the ventral tegmental area encode
individual differences in motivational “wanting” for reward cues. J. Neurosci. 40:8951–63
Fibiger HC, Phillips AG. 1986. Reward, motivation, cognition: psychobiology of mesotelencephalic dopamine
systems. In Handbook of Physiology: The Nervous System IV, ed. FE Bloom, SD Geiger, pp. 647–58. Bethesda,
MD: Am. Physiol. Soc.
Fourquet NY, Patterson TK, Li C, Castel AD, Knowlton BJ. 2020. Effects of age-related stereotype threat on
metacognition. Front. Psychol. 11:604978
Friedman MC, McGillivray S, Murayama K, Castel AD. 2015. Memory for medication side effects in younger
and older adults: the role of subjective and objective importance. Mem. Cogn. 43:206–15
Galli G, Sirota M, Gruber MJ, Ivanof BE, Ganesh J, et al. 2018. Learning facts during aging: the benets of
curiosity. Exp. Aging Res. 44:311–28
Galván A, Hare TA, Parra CE, Penn J, Voss H, et al. 2006. Earlier development of the accumbens relative to
orbitofrontal cortex might underlie risk-taking behavior in adolescents. J. Neurosci. 26:6885–92
Gardiner JM, Gawlik B, Richardson-Klavehn A. 1994. Maintenance rehearsal affects knowing, not remem-
bering; elaborative rehearsal affects remembering, not knowing. Psychon. Bull. Rev. 1:107–10
Garrett DD, Grady CL, Hasher L. 2010. Everyday memory compensation: the impact of cognitive reserve,
subjective memory, and stress. Psychol. Aging 25:74–83
Gasbarri A, Sulli A, Packard MG. 1997. The dopaminergic mesencephalic projections to the hippocampal
formation in the rat. Prog. Neuropsychopharmacol. Biol. Psychiatry 21:1–22
Gasbarri A, Verney C, Innocenzi R, Campana E, Pacitti C. 1994. Mesolimbic dopamine neurons innervating
the hippocampal formation in the rat: a combined retrograde tracing and immunohistochemical study.
Brain Res. 668:71–79
Ghetti S, Bunge SA. 2012. Neural changes underlying the development of episodic memory during middle
childhood. Dev. Cogn. Neurosci. 2:381–95
Giedd JN. 2004. Structural magnetic resonance imaging of the adolescent brain.Ann. NY Acad. Sci. 1021:77–85
Glimscher PW. 2011. Understanding dopamine and reinforcement learning: the dopamine reward prediction
error hypothesis. PNAS 108:15647–54
Gogtay N, Nugent TF 3rd, Herman DH, Ordonez A, Greenstein D, et al. 2006. Dynamic mapping of normal
human hippocampal development. Hippocampus 16:664–72
Grifn ML, Benjamin AS, Sahakyan L,Stanley SE.2019. A matter of priorities: High working memory enables
(slightly) superior value-directed remembering.J. Mem. Lang. 108:104032
Gruber MJ, Ritchey M, Wang S-F, Doss MK, Ranganath CR. 2016. Post-learning hippocampal dynamics
promote preferential retention of rewarding events. Neuron 89:1110–20
Hadwin AF, Winne PH, Stockley DB, Nesbit JC, Woszczyna C. 2001. Context moderates students’ self-
reports about how they study. J. Ed uc. Psychol. 93:477–87
Hanten G, Li X, Chapman SB, Swank P, Gamino JF, et al. 2007. Development of verbal selective learning.
Dev. Neuropsychol. 32:585–96
Hargis MB, Castel AD. 2017. Younger and older adults’ associative memory for social information: the role
of information importance. Psychol. Aging 32(4):325–30
Hargis MB, Castel AD. 2018. Younger and older adults’ associative memory for medication interactions of
varying severity. Memory 26(8):1151–58
Hargis MB, Siegel ALM, Castel AD. 2019. Motivated memory, learning, and decision making in older age:
shifts in priorities and goals. In The Aging Brain, ed. G Samanez-Larkin, pp. 135–64. Washington, DC:
Am. Psychol. Assoc.
Hargis MB, Whatley MC, Siegel ALM, Castel AD. 2020. Motivated cognition and curiosity in the aging
consumer. In The Aging Consumer, ed. A Drolet, C Yoon, pp. 47–66. New York: Routledge
Harvey DY, Wei T, Ellmore TM, Hamilton AC, Schnur TT. 2013. Neuropsychological evidence for the
functional role of the uncinate fasciculus in semantic control. Neuropsychologia 51:789–801
15.24 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
Hayes MG, Kelly AJ, Smith AD. 2013. Working memory and the strategic control of attention in older and
younger adults. J. Gerontol. Ser. B Psychol. Sci. Soc. Sci. 68:176–83
Hennessee JP, Castel AD, Knowlton BJ. 2017. Recognizing what matters: Value improves recognition by
selectively enhancing recollection. J. Mem. Lang. 94:195–205
Hennessee JP, Knowlton BJ, Castel AD. 2018. The effects of value on context-item associative memory in
younger and older adults. Psychol. Aging 33:46–56
Hennessee JP, Patterson TK, Castel AD, Knowlton BJ. 2019a. Forget me not: encoding processes in value-
directed remembering. J. Mem. Lang. 106:29–39
Hennessee JP, Reggente N, Cohen MS, Rissman J, Castel AD, Knowlton BJ. 2019b. White matter integrity in
brain structures supporting semantic processing is associated with value-directed remembering in older
adults. Neuropsychologia 129:246–54
Hertzog C, Dunlosky J. 1996. The aging of practical memory: an overview.BasicAppl.Mem.Res.TheoryContext
1:337–58
Hertzog C, Price J, Dunlosky J. 2008. How is knowledge generated about memory encoding strategy effec-
tiveness? Learn. Individ. Differ. 18:430–45
Hertzog C, Sinclair SM, Dunlosky J. 2010. Age differences in the monitoring of learning: cross-sectional
evidence of spared resolution across the adult life span. Dev. Psychol. 46:939–48
Hess TM, Growney CM, O’Brien EL, Neupert SD, Sherwood A. 2018. The role of cognitive costs, atti-
tudes about aging, and intrinsic motivation in predicting engagement in everyday activities. Psychol.Aging
33:953–64
Hidi SE, Renninger KA. 2019. Interest development and its relation to curiosity: needed neuroscientic re-
search. Educ. Psychol. Rev. 31:833–52
Isaacson RM, Fujita F. 2006. Metacognitive knowledge monitoring and self-regulated learning: academic suc-
cess and reections on learning. Int. J. Scholarsh. Teach. Learn. 6:39–55
Jay TM. 2003. Dopamine: a potential substrate for synaptic plasticity and memory mechanisms. Prog. Neurobiol.
69:375–90
Kim A, Merriam SB. 2004. Motivations for learning among older adults in a learning in retirement institute.
Educ. Gerontol. 30:441–55
Kinzer A, Suhr JA. 2016. Dementia worry and its relationship to dementia exposure, psychological factors,
and subjective memory concerns. Appl. Neuropsychol. Adult 23:196–204
Kornell N, Bjork RA. 2007. The promise and perils of self-regulated study. Psychon. Bull. Rev. 14:219–24
Kosobud AE, Harris GC, Chapin JK.1994. Behavioral associations of neuronal activity in theventral tegmen-
talareaoftherat.J. Neurosci. 14:7117–29
Leanos S, Kürüm E, Strickland-Hughes CM, Ditta AS, Nguyen G, et al. 2020. The impact of learning multiple
real-world skills on cognitive abilities and functional independence in healthy older adults. J. Gerontol.
Ser. B Psychol. Sci. Soc. Sci. 75:1155–69
Lebel C, Beaulieu C. 2011. Longitudinal development of human brain wiring continues from childhood into
adulthood. J. Neurosci. 31:10937–47
Li P, Zhang Y, Li W, Li X. 2018. Age-related differences in effectiveness of item restudy choices: the role of
value. Aging Neuropsychol. Cogn. 25:122–31
Lisman J, Grace AA. 2005. The hippocampal-VTA loop: controlling the entry of information into long-term
memory. Neuron 46:703–13
Lisman J, Grace AA, Duzel E. 2011. A neo-Hebbian framework for episodic memory; role of dopamine-
dependent late LTP. Trends Neurosci. 34:536–47
Ljungberg T, Apicella P, Schultz W. 1992. Responses of monkey dopamine neurons during learning of be-
havioral reactions. J. Neurophysiol. 67:145–63
Lo JC, Bennion KA, Chee MW. 2016. Sleep restriction can attenuate prioritization benets on declarative
memory consolidation. J. Sleep Res. 25:664–72
Madan CR. 2017. Motivated cognition: effects of reward, emotion, and other motivational factors across a
variety of cognitive domains. Collabra Psychol. 3(1):24
Madan CR, Fujiwara E, Gerson BC,Caplan JB. 2012. High reward makes items easier to remember, but harder
to bind to a new temporal context. Front. Integr. Neurosci. 6:61
www.annualreviews.org •Memory and Reward-Based Learning 15.25
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
Mather M. 2016. The affective neuroscience of aging. Annu. Rev. Psychol. 67:213–38
Mazerolle M, Régner I, Barber SJ, Paccalin M, Miazola AC, et al. 2017. Negative aging stereotypes impair
performance on brief cognitive tests used to screen for predementia. J. Gerontol. Ser. B Psychol. Sci. Soc.
Sci. 72(6):932–36
McDaniel MA, Einstein GO. 2020. Training learning strategies to promote self-regulation and transfer: the
knowledge, belief, commitment, and planning framework. Perspect. Psychol. Sci. 15:1363–81
McGillivray S, Castel AD. 2011. Betting on memory leads to metacognitive improvement by younger and
older adults. Psychol. Aging 26:137–42
McGillivray S, Castel AD.2017. Older and younger adults’ strategic control of metacognitive monitoring: the
role of consequences, task experience and prior knowledge. Exp. Aging Res. 43:233–56
McGillivray S, Murayama K, Castel AD. 2015. Thirst for knowledge: the effects of curiosity and interest on
memory in younger and older adults. Psychol. Aging 30:835–41
Middlebrooks CD, Castel AD. 2018. Self-regulated learning of important information under sequential and
simultaneous encoding conditions. J. Exp. Psychol. Learn. Mem.Cogn. 44:779–92
Middlebrooks CD, Kerr T, Castel AD. 2017. Selectively distracted: divided attention and memory for impor-
tant information. Psychol. Sci. 28:1103–15
Middlebrooks CD, Murayama K, Castel AD. 2016. The value in rushing: memory and selectivity when short
on time. Acta Psychol. 170:1–9
Moscovitch M, Cabeza R, Winocur G, Nadel L. 2016. Episodic memory and beyond: the hippocampus and
neocortex in transformation. Annu. Rev. Psychol. 67:105–34
Murayama K, Kitagami S. 2014. Consolidation power of extrinsic rewards: Reward cues enhance long-term
memory for irrelevant past events.J.Exp.Psychol.Gen.143:15–20
Murphy DH, Agadzhanyan K, Whatley MC, Castel AD. 2021. Metacognition and uid intelligence in value-
directed remembering. Metacogn. Learn. https://doi.org/10.1007/s11409-021-09265- 9
Murphy DH, Castel AD. 2020. Responsible remembering: how metacognition impacts adaptive selective
memory. Z. Psychol. 228:301–3
Murphy DH, Castel AD. 2021. Responsible remembering and forgetting as contributors to memory for im-
portant information. Mem. Cogn. 49:895–911
Naveh-Benjamin M. 2000. Adult age differences in memory performance: tests of an associative decit hy-
pothesis. J. Exp. Psychol. Learn. Mem. Cogn. 26:1170–87
Nguyen C, Leanos S, Natsuaki MN, Rebok GW, Wu R. 2020. Adaptation for growth via learning new skills
as a means to long-term functional independence in older adulthood: insights from emerging adulthood.
Gerontologist 60:4–11
Nguyen LT, Marini F, Shende SA, Llano DA, Mudar RA. 2020. Investigating EEG theta and alpha oscillations
as measures of value-directed strategic processing in cognitively normal younger and older adults. Behav.
Brain Res. 391:112702
Nguyen LT, Marini F, Zacharczuk L, Llano DA, Mudar RA. 2019. Theta and alpha band oscillations during
value-directed strategic processing. Behav. Brain Res. 367:210–14
Nyberg L, Pudas S. 2019. Successful memory aging. Annu. Rev. Psychol. 70:219–43
Ofen N, Kao YC, Sokol-Hessner P, Kim H, Whiteld-Gabrieli S, Gabrieli JD. 2007. Development of the
declarative memory system in the human brain. Nat. Neurosci. 10:1198–205
Ozubko JD, Gopie N, MacLeod CM. 2012. Production benets both recollection and familiarity. Mem. Cogn.
40:326–38
Pintrich PR. 1995. Understanding self-regulated learning. New Dir. Teach. Learn. 63:3–12
Polich J. 2007. Updating P300: an integrative theory of P3a and P3b. Clin. Neurophysiol. 118:2128–48
Popov V, Marevic I, Rummer J, Reder LM. 2019. Forgetting is a feature, not a bug: Intentionally forgetting
some things helps us remember others by freeing up working memory resources. Psychol. Sci. 30:1303–17
Reggente N, Cohen MS, Zheng ZS, Castel AD, Knowlton BJ, Rissman J. 2018. Memory recall for high re-
ward value items correlates with individual differences in white matter pathways associated with reward
processing and fronto-temporal communication. Front. Hum. Neurosci. 12:241
Rescorla RA, Wagner AD. 1972. A theory of Pavlovian conditioning: variations in the effectiveness of rein-
forcement and nonreinforcement. In Classical Conditioning II: Current Research and Theory, ed. AH Black,
WF Prokasy, pp. 64–99. New York: Appleton, Century, Crofts
15.26 Knowlton •Castel
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
Robison MK, Unsworth N. 2017. Working memory capacity, strategic allocation of study time, and value-
directed remembering. J. Mem. Lang. 93:231–44
Sakaki M, Yagi A, Murayama K. 2018. Curiosity in old age: a possible key to achieving adaptive aging. Neurosci.
Biobehav. Rev. 88:106–16
Salthouse TA. 2019. Trajectories of normal cognitive aging. Psychol. Aging 34(1):17–24
Samanez-Larkin GR, Worthy DA, Mata R, McClure SM, Knutson B. 2014. Adult age differences in frontostri-
atal representation of prediction error but not reward outcome. Cogn. Affect. Behav. Neurosci. 14:672–82
Schultz W. 2007. Multiple dopamine functions at different time-courses. Annu. Rev. Neurosci. 30:259–88
Schultz W. 2015. Neuronal reward and decision signals: from theories to data. Physiol. Rev. 95:853–951
Schultz W. 2016. Dopamine reward prediction error coding. Dialogues Clin. Neurosci. 18:23–32
Schwartz ST, Siegel ALM, Castel AD. 2020. Strategic encoding and enhanced memory for positive value-
location associations. Mem. Cogn. 48:1015–31
Shohamy D, Adcock RA. 2010. Dopamine and adaptive memory. Trends Cogn. Sci. 14:464–72
Siegel ALM, Castel AD. 2018. Memory for important item-location associations in younger and older adults.
Psychol. Aging 33:30–45
Siegel ALM, Castel AD. 2019. Age-related differences in metacognition for memory capacity and selectivity.
Memory 27:1236–49
Siegel ALM, Whatley MC, Hargis MB, Castel AD. 2020. Changes in memory and metacognition in older
adulthood. In The Aging Consumer, ed. A Drolet, C Yoon, pp. 26–46. New York: Routledge
Soderstrom NC, Bjork RA. 2014. Testing facilitates the regulation of subsequent study time. J. Mem. Lang.
73:99–115
Spaniol J, Bowen HJ, Wegier P, Grady C. 2015. Neural responses to monetary incentives in younger and older
adults. Brain Res. 1612:70–82
Spaniol J, Schain C, Bowen HJ. 2014. Reward-enhanced memory in younger and older adults. J. Gerontol. Ser.
B Psychol. Sci. Soc. Sci. 69:730–40
Stefanidi A, Ellis DM, Brewer GA. 2018. Free recall dynamics in value-directed remembering. J. Mem. Lang.
100:18–31
Storm BC, Hickman ML, Bjork EL. 2016. Improving encoding strategies as a function of test knowledge and
experience. Mem. Cogn. 44:660–70
Swirsky LT, Spaniol J. 2019. Cognitive and motivational selectivity in healthy aging. Wiley Interdiscip. Rev.
Cogn. Sci. 10:e1512
Thomas AK, Gutchess A, eds. 2020. The Cambridge Handbook of Cognitive Aging: A Life Course Perspective.
Cambridge, UK: Cambridge Univ. Press
Tulving E, Arbuckle TY. 1966. Input and output interference in short-term associative memory. J. Ex p. Psychol.
72:145–50
Unsworth N. 2010. On the division of working memory and long-term memory and their relation to intelli-
gence: a latent variable approach. Acta Psychol. 134:16–28
Unsworth N. 2016. Working memory capacity and recall from long-term memory: examining the inuences
of encoding strategies, study time allocation, search efciency, and monitoring abilities. J. Exp. Psychol.
Learn. Mem.Cogn. 42:50–61
Villaseñor JJ, Sklenar AM, Frankenstein AN, Urban Levy P, McCurdy MP, Leshikar ED. 2021. Value-
directed memory effects on item and context memory. Mem. Cogn. https://doi.org/10.3758/s13421-
021-01153-6
Visser M, Jeffries E, Lambon Ralph MA. 2010. Semantic processing in the anterior temporal lobes: a meta-
analysis of the functional neuroimaging literature. J. Cogn. Neurosci. 22:1083–94
Wendelken C, O’Hare ED, Whitaker KJ, Ferrer E, Bunge SA. 2011. Increased functional selectivity over
development in rostrolateral prefrontal cortex.J.Neurosci.31:17260–68
Witherby AE, Tauber SK, Rhodes MG, Castel AD. 2019. Aging and forgetting: Forgotten information is
perceived as less important than is remembered information. Psychol. Aging 34:228–41
Wittmann BC, Bunzeck N, Dolan RJ, Düzel E. 2007. Anticipation of novelty recruits reward system and
hippocampus while promoting recollection. Neuroimage 38:194–202
www.annualreviews.org •Memory and Reward-Based Learning 15.27
PS73CH15_Knowlton ARjats.cls September 21, 2021 9:22
Wong S, Irish M, Savage G, Hodges JR, Piguet O, Hornberger M. 2019. Strategic value-directed learning
and memory in Alzheimer’s disease and behavioural-variant frontotemporal dementia. J. Neuropsychol.
13:328–53
Yarkoni T, Poldrack RA, Nichols TE, Van Essen DC, Wager TD. 2011. Large-scale automated synthesis of
human functional neuroimaging data. Nat. Methods 8:665–70
Yonelinas AP. 2002. The nature of recollection and familiarity: a review of 30 years of research. J. Mem. Lang.
46:441–517
Zanto TP, Gazzaley A. 2019. Aging of the frontal lobe. Handb.Clin. Neurol. 163:369–89
15.28 Knowlton •Castel
