Fig 4 - uploaded by Thomas R Zentall
Content may be subject to copyright.
Design of Experiment 2. The left panel shows a DRO trial with completion of the DRO requirement followed by one simultaneous discrimination (red+ yel- low2). The right panel shows a FI trial with completion of the FI requirement followed by a different simultaneous discrimination (green+ blue2). Colors associated with the two simultaneous discriminations were counterbalanced.
Source publication
Several types of contrast effects have been identified including incentive contrast, anticipatory contrast, and behavioral contrast. Clement, Feltus, Kaiser, and Zentall (2000) proposed a type of contrast that appears to be different from these others and called it within-trial contrast. In this form of contrast the relative value of a reinforcer d...
Context in source publication
Context 1
... of the positive discriminative stimulus (e.g., green) resulted in reinforcement and the ITI, whereas choice of the negative discriminative stimulus resulted in the ITI alone. For 4 of the pigeons, red and yellow stimuli followed the DRO schedule and green and blue stimuli followed the FI schedule (see Figure 4). For the remaining 3 pigeons, red and yellow stimuli followed the FI schedule and green and blue stimuli followed the DRO schedule. ...
Similar publications
Restorativeness is defined as the potential of the environment to re-establish certain cognitive capacities related to human information processing. The most frequently used instrument for evaluating the restorativeness of places is the Perceived Restorativeness Scale, proposed by Hartig et al. (1991). Later on, shorter versions of the Perceived Re...
This experiment investigated relational complexity and relational shift in judgments of auditory patterns. Pitch and duration values were used to construct two-note perceptually similar sequences (unary relations) and four-note relationally similar sequences (binary relations). It was hypothesized that 5-, 8- and 11-year-old children would perform...
Three experiments with 144 female undergraduates tested the hypothesis that field-independent (FI) Ss would be better reality monitors and would be better able to separate the self from the nonself than field-dependent (FD) Ss. It is noted that, according to M. K. Johnson and C. L. Raye (see record
1981-06694-001), reality monitoring is the proces...
![Figure 8. Spectral envelopes [amplitudes (in dB) v. frequency (in Hz)]...](profile/Michael-Hall-5/publication/277205401/figure/fig4/AS:964476157308929@1606960443311/Spectral-envelopes-amplitudes-in-dB-v-frequency-in-Hz-for-the-resynthesized-violin_Q320.jpg)
Classic studies based on multi-dimensional scaling of dissimilarity judgments, and on discrimination, for musical instrument sounds have provided converging support for the importance of relatively static, spectral cues to timbre (e.g., energy in the higher harmonics, which has been associated with perceived brightness), as well as dynamic, tempora...
In this work, we consider optimal state discrimination for a quantum system that interacts with an environment, i.e., states evolve under a quantum channel. We show the conditions on a quantum channel and an ensemble of states such that a measurement for optimal state discrimination is preserved. In particular, we show that when an ensemble of stat...
Citations
... Wang et al., 2017). For example, animals tend to prefer outcomes that have previously been associated with more effortful behavior (Klein et al., 2005;Singer et al., 2007;Tsukamoto et al., 2017). Returning to our earlier example, these frameworks predict that the views atop the mountain would be more rewarding if we chose to climb to the peak rather than take the chairlift. ...
Humans routinely learn the value of actions by updating their expectations based on past outcomes - a process driven by reward prediction errors (RPEs). Importantly, however, implementing a course of action also requires the investment of effort. Recent work has revealed a close link between the neural signals involved in effort exertion and those underpinning reward-based learning, but the behavioural relationship between these two functions remains unclear. Across two experiments, we tested healthy male and female human participants (N=140) on a reinforcement learning task in which they registered their responses by applying physical force to a pair of hand-held dynamometers. We examined the effect of effort on learning by systematically manipulating the amount of force required to register a response during the task. Our key finding, replicated across both experiments, was that greater effort increased learning rates following positive outcomes and decreased them following negative outcomes, which corresponded to a differential effect of effort in boosting positive RPEs and blunting negative RPEs. Interestingly, this effect was most pronounced in individuals who were more averse to effort in the first place, raising the possibility that the investment of effort may have an adaptive effect on learning in those less motivated to exert it. By integrating principles of reinforcement learning with neuroeconomic approaches to value-based decision making, we show that the very act of investing effort modulates one's capacity to learn, and demonstrate how these functions may operate within a common computational framework.SIGNIFICANCE STATEMENTRecent work suggests that learning and effort may share common neurophysiological substrates. This raises the possibility that the very act of investing effort influences learning. Here, we tested whether effort modulates teaching signals in a reinforcement learning paradigm. Our results showed that effort resulted in more efficient learning from positive outcomes and less efficient learning from negative outcomes. Interestingly, this effect varied across individuals, and was more pronounced in those who were more averse to investing effort in the first place. These data highlight the importance of motivational factors in a common framework of reward-based learning, which integrates the computational principles of reinforcement learning with those of value-based decision-making.
... That is, relative to the total duration of the trial, it appeared relatively closer to reinforcement. Singer et al. (2007) tested this hypothesis by controlling for the total duration of the trial. They preceded one color with a fixed interval schedule (that required pecking) and the other color with a differential reinforcement of other behavior schedule (the same interval that required not pecking). ...
Pavlovian processes are likely responsible for the varied contexts in which contrast occurs between what is expected and what is obtained. Such contrast effects result in paradoxical biases and even suboptimal choice by animals. For example, pigeons prefer a suboptimal alternative that results in a stimulus signaling a low probability (20%) high reward (ten pellets) over an optimal alternative that results in a stimulus signaling a high probability (100%) of a smaller reward (three pellets). This effect is analogous to human unskilled gambling. In another case, pigeons prefer a stimulus that has required many pecks to obtain over one the has required one peck to obtain (a so-called justification of effort effect). In a third line of research that investigated the preference for risky choice over safe choice, pigeons chose between two alternatives, a safe choice that resulted in two pellets of food, or a risky choice that resulted in either one or three pellets of food. In general, the pigeons preferred the risky alternative, but importantly, their choice was influenced by whether the choice reflected a gain or a loss - the difference between what was shown to them (one, two, or three pellets) and what they received. Each of these lines of research suggest the importance of contrast effects produced by Pavlovian processes that result in biases or suboptimal behavior.
... The role of Δ in the hypothesis is very similar to the within-trial contrast idea proposed by Zentall (2005). According to this idea, the hedonic state of a subject changes between the end of a less appetitive event and a stimulus signaling reward or the reward itself (see Singer et al., 2007). In the standard suboptimal choice task, the preferred alternative has a Δ 1 = 1 (p 1,1 = 1 and p 1,2 = 0), the maximum possible difference; the other alternative has a Δ 2 = 0 (p 2,1 = p 2,2 = .5), ...
In a concurrent‐chain procedure, pigeons choose between 2 initial‐link stimuli; one is followed by terminal link stimuli that signal reliably whether food will be delivered after a delay; the other is followed by terminal link stimuli that do not signal whether food will be delivered after the delay. Pigeons prefer the former alternative even when it yields a lower overall probability of food. Recently, we proposed the Delta‐Sigma (∆‐∑) hypothesis to explain the effect: Preference depends on the difference (∆) between the reinforcement probabilities associated with the terminal link stimuli, and the overall probability of reinforcement (∑) associated with the alternative. The hypothesis predicts that, for constant ∑, animals should prefer alternatives with greater ∆ values regardless of the specific probabilities of reinforcement that determine ∆. In 2 experiments, we tested this prediction by comparing a ∆ = .5 against a ∆ = 0 alternative, with the former obtained with different pairs of reinforcement probabilities across conditions. The results supported the hypothesis when the 2 probabilities defining ∆ were significantly greater than 0, but not when one of them was close to 0. The results challenge our theoretical accounts of suboptimal choice and the variables considered to determine pigeons’ preference.
... To test more directly the hypothesis that delay reduction theory can account for the within-trial contrast effect, Singer, Berry, and Zentall (2007) used a procedure in which on some trials the pigeons were given a fixed interval (FI) 20 s schedule prior presentation of one simultaneous discrimination (the first response after 20 s presented the simultaneous discrimination; see Figure 5). On the remaining trials, the pigeons were given a differential reinforcement of other behavior (DRO) 20 s schedule (the pigeons had to refrain from pecking the light for 20 s on those trials). ...
When humans make biased or suboptimal choices, they are often attributed to complex cognitive processes that are viewed as being uniquely human. Alternatively, several phenomena, such as suboptimal gambling behavior and cognitive dissonance (justification of effort) may be explained more simply as examples of the contrast between what is expected and what occurs as well as Wagner's Standard Operating Procedure model based on reward prediction error. For example, when pigeons are attracted to choices involving a suboptimal, low probability of a high payoff, as in unskilled gambling behavior, it may be attributed to reward prediction error or the contrast between the low probability of reward expected and the sometimes high probability of reward obtained (when one wins). Similarly, justification of effort, the tendency to attribute greater value to rewards that are difficult to obtain, is typically explained in terms of the tendency to inflate the value of a reward to justify the effort required to obtain it. When pigeons prefer outcomes that require more effort to obtain, however, it is more likely to be explained in terms of contrast between the effort and the reward that follows. We readily attribute the behavior of animals to contrast-like effects or reward prediction error, however, when similar behavior occurs in humans, we also should be prepared to explain it in terms of simpler learning mechanisms. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
... In lay terms, stimuli that predict a more difficult task become less preferred over repeated exposures, and stimuli that follow a more difficult task become more preferred over repeated exposures. This phenomenon has been demonstrated with animals (e.g., pigeons, locusts, rats, and starlings; Meindl, 2012) and humans (e.g., Klein, Bhatt, & Zentall, 2005), as well as with many different types of effort tasks such as high versus low effort (e.g., Alessandri, Darchville, & Zentall, 2008), high versus low probabilities of reinforcement (e.g., Clement & Zentall, 2002), short versus long delays to the terminal component (e.g., Alessandri et al., 2008), preferred versus less preferred schedules of reinforcement (Singer, Berry, & Zentall, 2007), the absence or presence of reinforcement (Friedrich, Clement, & Zentall, 2005), and low versus high states of food deprivation (e.g., Aw, Holbrook, de Perera, & Kacelnik, 2009;see Meindl, 2012, for a literature review). ...
Researchers have examined preference for the format of delivery of feedback, however little research has examined strategies to increase feedback and praise desirability. The current study aimed to evaluate whether preference shifted for stimuli that signaled work and for manager-praise stimuli that were delivered after work. Moreover, we sought to understand if these preference shifts were greater when the work was higher-effort compared to lower-effort work tasks (a phenomenon described in the basic literature as within-trial contrast). The study took place in a simulated work environment. Sixteen university students clicked on a shape that produced the work task, completed a mock medical data-entry task, and received manager-praise on an FR1 and FR20 schedule. At the start of the study and following each contingency exposure trials, they completed a preference assessment for both the shapes and the manager-feedback stimuli. The primary dependent variable was the percentage of preference change for two categories of stimuli – shapes presented at the beginning of trials that signaled which condition participants were in and manager praise stimuli presented at the end of trials. Preference for the shape stimuli for both high- and low-effort stimuli decreased indicating that stimuli that signal work may become less preferred. Preference for the manager-praise stimuli for both high and low-effort stimuli increased suggesting that stimuli that follow work may become more preferred. Overall, these data
suggest that the conditions under which managers deliver feedback and praise may influence preference for those stimuli.
... The purpose of the Singer et al. (2007) experiment was to create trials of equal duration, to determine if the subjects had a preference for the stimulus that followed each of the schedules. The purpose of the present experiment was to more directly investigate relative schedule preference between a DRO and FI schedule in which schedule duration was controlled by yoking the schedules on a trial-by-trial basis. ...
... A preference for the FI schedule over the DRO schedule would suggest that pigeons prefer pecking rather than being required to refrain from pecking. In an FI schedule, a peck is required immediately before reinforcement and the association of that peck with reinforcement could actually result in a preference for pecking (as Singer et al., 2007, found for one pigeon). The FI schedule may also be preferred because, although the schedules are equated for their duration, pecking has been found to result in the subjective speeding up of the passage of time (Zentall & Singer, 2008). ...
... Individual differences in our data are in line with Singer, Berry, and Zentall (2007). Singer et al. found individual differences in schedule preference (DRO vs. FI) when the delay to reinforcement was controlled, as it was in the present experiment. ...
Animals are expected to minimize time and effort to reinforcement. Thus, not pecking should be preferred over pecking. However, even if time is held constant, pigeons often peck when it is allowed but not required (e.g., fixed time schedules), but with such schedules pecking may be adventitiously reinforced. In the present experiment, to better compare a schedule of reinforcement that requires pecking with one that requires the absence of pecking, we compared a modified fixed-interval (FI) schedule in which reinforcement follows the first peck after the interval has elapsed and a differential-reinforcement-of-other behavior (DRO) schedule, which requires pigeons to abstain from pecking for a similar interval. The delay to reinforcement was matched on a trial-by-trial basis by yoking the duration of the FI to match the DRO schedule that preceded it. Of 12 pigeons, six preferred the DRO schedule over the FI schedule and six did not show a schedule preference. Those that were indifferent between the schedules apparently had a stronger spatial preference than their schedule preference. Individual differences in the preference of the pigeons may have been related to their behavior during the DRO schedule.
... Theoretically, the contrast effect enhances the perceived value of that option, and hence it is preferred to an otherwise equivalent option but with no prior effort. This preference has also been observed for options that followed the absence of reward (Friedrich, Clement, & Zentall, 2005) or a delay (DiGian, Friedrich, & Zentall, 2004); or for a stimulus that has been previously identified as the less preferred (Singer, Berry, & Zentall, 2007). As a behavioral phenomenon, the effect has not always been replicated (e.g., Arantes & Grace, 2008;Vasconcelos, Urcuioli, & Lionello-DeNolf, 2007). ...
The sunk cost effect is the bias or tendency to persist in a course of action due to prior investments of effort, money or time. At the time of the only review on the sunk cost effect across species (Arkes & Ayton, 1999), research with nonhuman animals had been ecological in its nature, and the findings about the effect of past investments on current choice were inconclusive. However, in the last decade a new line of experimental laboratory-based research has emerged with the promise of revolutionizing the way we approach the study of the sunk cost effect in nonhumans. In the present review we challenge Arkes and Ayton's conclusion that the sunk cost effect is exclusive to humans, and describe evidence for the sunk cost effect in nonhuman animals. By doing so, we also challenge the current explanations for the sunk cost effect in humans, as they are not applicable to nonhumans. We argue that a unified theory is called for, because different independent variables, in particular, investment amount, have the same influence on the sunk cost effect across species. Finally, we suggest possible psychological mechanisms shared across different species, contrast and depreciation, that could explain the sunk cost effect.
... Contrast and the SiGN hypothesis make very similar predictions, which is not surprising given the similarity in their analyses. Attempts to distinguish between the contrast account and DRT (Fantino, 1969), which initially inspired the SiGN hypothesis, have been complicated by the difference in the procedures used by the two approaches (Singer, Berry, & Zentall, 2007;Singer & Zentall, 2011). The SiGN hypothesis differs from DRT in that it focuses on local context effects, a focus that is shared with the contrast model. ...
Pigeons and other animals sometimes deviate from optimal choice behavior when given informative signals for delayed outcomes. For example, when pigeons are given a choice between an alternative that always leads to food after a delay and an alternative that leads to food only half of the time after a delay, preference changes dramatically depending on whether the stimuli during the delays are correlated with (signal) the outcomes or not. With signaled outcomes, pigeons show a much greater preference for the suboptimal alternative than with unsignaled outcomes. Key variables and research findings related to this phenomenon are reviewed, including the effects of durations of the choice and delay periods, probability of reinforcement, and gaps in the signal. We interpret the available evidence as reflecting a preference induced by signals for good news in a context of uncertainty. Other explanations are briefly summarized and compared.
... That is, strong positive contrast was assumed to occur between the probability of reinforcement associated with choice of the suboptimal alternative (20% reinforcement) and the probability of reinforcement associated with the conditioned reinforcer that followed (100%), whereas no contrast was expected to occur between the probability of reinforcement associated with choice of the optimal alternative (50% reinforcement) and the probability of reinforcement associated with the conditioned reinforcer that followed (50% reinforcement). The contrast account is similar to the delay reduction account, however, delay reduction is limited to the temporal dimension, whereas contrast is predicted to occur whenever a conditioned reinforcer predicts an improvement in conditions without necessarily predicting a reduction in delay to reinforcement (see Singer, Berry, & Zentall, 2007;Singer & Zentall, 2011). ...
When humans engage in commercial (totally probabilistic) gambling they are making suboptimal choices because the return is generally less than the investment. This review (a) examines the literature on pigeon suboptimal choice, (b) describes the conditions under which it occurs, (c) identifies the mechanisms that appear to be responsible for the effect, and (d) suggests that similar processes may be able to account for analogous suboptimal choice when humans engage in commercial gambling. Pigeons show suboptimal choice when they choose between 1 alternative that 20% of the time provides them with a signal that they will always get fed or 80% of the time with a signal that they will not get fed (overall 20% reinforcement) and a second alternative that 100% of the time provides them with a signal that they will get fed 50% of the time (overall 50% reinforcement). The pigeons' strong preference for the suboptimal choice was investigated in a series of experiments that found the preference for the suboptimal alternative was determined by the value of the signal that predicted reinforcement, rather its frequency and that the frequency of the signal that predicted nonreinforcement had little effect on the suboptimal choice. Paradoxically, this account makes the prediction that pigeons will be indifferent between an alternative that 50% of the time provides a fully predictive stimulus for reinforcement and an alternative that 100% of the time provides a fully predictive stimulus for reinforcement. The similarities and differences of this suboptimal choice task to human gambling are discussed. (PsycINFO Database Record
... In other words, pigeons preferred the color associated with more effort, although rewards for correct choices (of red, or green) were the same. This preference has been observed in starlings (Aw et al., 2011, Experiment 3) and occurs for options that follow the absence of reward (Friedrich et al., 2005) or a delay (DiGian et al., 2004), or for a stimulus that has been previously identified as the less preferred (Singer et al., 2007). The effect has not always been replicated (e.g., Arantes and Grace, 2008a; Vasconcelos et al., 2007a ), perhaps because lower probability of reward for correct choices in training may reduce the size of the effect (Gipson et al., 2009). ...
