Figure - available from: Animal Cognition
This content is subject to copyright. Terms and conditions apply.
Percentage choice of the suboptimal alternative by pigeons in (1) Group 50% that chose between one alternative that provided 50% reinforcement and the other alternative that provided 100% reinforcement and (2) Group 75% that chose between one alternative that provided 75% reinforcement and the other alternative that provided 100% reinforcement
Source publication
When pigeons are given a choice between 50% signaled reinforcement and 100% reinforcement they typically do not choose optimally, sometimes even preferring 50% reinforcement. Smith and Zentall (J Exp Psychol Anim Behav Process 42:212–220, 2016) proposed that choice depends primarily on the predictive value of the signal for reinforcement associated...
Citations
... Therefore, the contrast for S-was minimal and had little to no influence in preference. More recent research suggests, however, that negative contrast might have an impact if it is sufficiently large (Zentall et al., 2019). Applying this argument to the results presented here, we found that, for the partial reinforcement of S-, the expectation of overall reinforcement became .6, ...
The present study evaluated the role of inhibition in paradoxical choice in pigeons. In a paradoxical choice procedure, pigeons receive a choice between two alternatives. Choosing the "suboptimal" alternative is followed 20% of the time by one cue (the S+) that is always reinforced, and 80% of the time by another cue (S-) that is never reinforced. Thus, this alternative leads to an overall reinforcement rate of 20%. Choosing the "optimal" alternative, however, is followed by one of two cues (S3 or S4), each reinforced 50% of the time. Thus, this alternative leads to an overall reinforcement rate of 50%. González and Blaisdell (2021) reported that development of paradoxical choice was positively correlated to the development of inhibition to the S- (signal that no food will be delivered on that trial) post-choice stimulus. The current experiment tested the hypothesis that inhibition to a post-choice stimulus is causally related to suboptimal preference. Following acquisition of suboptimal preference, pigeons received two manipulations: in one condition one of the cues in the optimal alternative (S4) was extinguished and, in another condition, the S- cue was partially reinforced. When tested on the choice task afterward, both manipulations resulted in a decrement in suboptimal preference. This result is paradoxical given that both manipulations made the suboptimal alternative the richer option. We discuss the implications of our results, arguing that inhibition of a post-choice cue increases attraction to or value of that choice.
... Content may be shared at no cost, but any requests to reuse this content in part or whole must go through the American Psychological Association. 2 DUNN, PISKLAK, MCDEVITT, AND SPETCH 1974 article-specifically, choice between signaled 50% food (suboptimal alternative) and 100% food Case & Zentall, 2018;Gipson et al., 2009;Kendall, 1985;McDevitt et al., 1997;Zentall et al., 2019). These procedures tend to foster a large amount of variation from subject to subject leading some to speculate that pigeons are indifferent between the alternatives and that the discrepant preferences, seen from pigeon to pigeon, reflect nothing more than biases produced by small uncontrolled aspects of the experiment . ...
... These results are shown in Figure 2. When the delay to food is short (e.g., less than 10 s), pigeons are likely to prefer the optimal alternative. However, with delays of 10 s or longer, mean choice proportions for the suboptimal alternative are typically above indifference, indicating a small but reliable preference for the suboptimal alternative (e.g., Case & Zentall, 2018;Zentall et al., 2019). Thus, any viable model of suboptimal choice would need to allow for preference of signaled 50% food over 100% food. ...
... Indirect evidence that forced exposures matter is provided by a comparison across studies that used the same parameters (FR 1 initial links, 10-s terminal links and choice between signaled 50% and 100%) but differed in forced exposures. Studies that did not include repeated forced exposures found relatively low choice proportions for the suboptimal alternative under these parameters, whereas studies with more forced-exposure trials than choice trials found choice proportions that were near or above 0.50 (Case & Zentall, 2018;Zentall et al., 2019;. Belke and Spetch (1994) provided asymmetrical forced-exposure to the suboptimal alternative by repeating any trial that ended in no food. ...
As first reported several decades ago, pigeons (Columba livia) sometimes choose options that provide less food over options that provide more food. This behavior has been variously referred to as suboptimal, maladaptive, or paradoxical because it lowers overall food intake. A great deal of research has been directed at understanding the conditions under which animals and people make suboptimal choices and the mechanisms that drive this behavior. Here, we review the literature on suboptimal choice and the variables that play a role in this phenomenon. Suboptimal choice is most likely to occur when the outcomes following a choice are uncertain, when the outcomes are delayed after the choice, and when the outcomes are signaled only on the option that provides food less often. We propose a mathematical formalization of the signal for good news (SiGN) model which assumes that a signal for a reduction in delay to food reinforces choice. We generate predictions from the model about the effect of parameters that characterize suboptimal choice and we show that, even in the absence of free parameters, the SiGN model provides a very good fit to the choice proportions of birds from a large set of conditions across studies from numerous researchers. R code for SiGN predictions and the data set are available on the Open Science Framework (https://osf.io/39qtj). We discuss limitations of the model, propose directions for future research, and discuss the general applicability of this research to understanding how rewards and signals for reward may combine to reinforce behavior.
... Nevertheless, when this information is not available or the environment is familiar, they are able to use olfactory and visual cues for their orientation in an arena (Gagliardo et al., 2001). Also, in traditional cue-response tasks, they can easily learn the meaning of 4-5 stimuli simultaneously and their learned performance is reliable in time (e.g., Wittek et al., 2021;Zentall et al., 2019). Thus, a deficiency in the ability of pigeons to follow visual cues is unlikely to account for deviations from their expected behavior. ...
One major survival-related activity of organisms is to seek food in their environment. To this end, they exploit previously rewarding locations and attempt to approach the cues predictive of the edible items they detect or expect. But foraging is unlikely to be a matter of reinforcement only. If it was, however, foraging activity should follow principles of extinction learning: It should be abolished in a location without reinforcement and proportionally be reduced in a partially reinforced location relative to a fully reinforced one containing the same number of food items. We tested these two hypotheses using a foraging board, which allowed pigeons to find food items hidden in perforated holes. Our results showed that the overall time spent and the overall number of pecks given in one area was related to reinforcement density in that area. To a lesser extent, the same phenomenon occurred with respect to the number of visits per area. However, the time-per-visit and pecks-per-visit ratios were higher in the partially vs. fully reinforced area, suggesting that the pigeons foraged more than expected when food was uncertain. These results will be discussed in the context of the matching law and optimal foraging.
... Under conditioned in which pigeons show a preference for 50% reinforcement over 100% reinforcement and the value of the positive conditioned stimuli is equated between the choice alternatives (at 100% reinforcement) contrast appears to be the mechanism responsible for the suboptimal choice (e.g., Case & Zentall, 2018;Vasconcelos et al., 2015). Furthermore, Zentall et al. (2019) found that under conditions similar to those of Case and Zentall (2018), if contrast is reduced by increasing the probability of reinforcement associated with the suboptimal alternative from 50% to 75%, preference for the suboptimal alternative is eliminated. That is, although the probability of reinforcement associated with the suboptimal alternative was increased from 50% to 75%, there was no longer a preference for that alternative. ...
Pigeons typically prefer a 20% probability of signaled reinforcement over a 50% probability of unsignaled reinforcement. There is even evidence that they prefer 50% signaled reinforcement over 100% reinforcement. It has been suggested that this effect results from contrast between the expected probability of reinforcement (e.g., 50%) at the time of choice and the value of the positive signal for reinforcement (100%). Alternatively, it is primarily the value of the positive signal for reinforcement itself that determines suboptimal choice. To attempt to distinguish between these two hypotheses, in Experiment 1, we gave pigeons a choice between (a) a 50% reinforcement alternative that was followed by one of two signals for 100% reinforcement, each 25% of the time, or a signal for the absence of reinforcement 50% of the time (50% contrast) and (b) a 25% reinforcement alternative that was followed by a signal for 100% reinforcement 25% of the time, or a signal for the absence of reinforcement 75% of the time (75% contrast). In spite of the difference in contrast, the pigeons were indifferent between the two alternatives. In Experiment 2, when contrast was held constant at 50% and the value of the positive signals for reinforcement were different, we found support for choice based on the value of the positive signal for reinforcement. Thus, it appears that pigeons' choice depends primarily on the value of the outcome rather than its frequency or contrast. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
... Spetch et al. (1990) did not include FE trials and found strong preference for the optimal alternative. In contrast, studies by Zentall and colleagues (Case & Zentall, 2018;Zentall et al., 2019) provided two FE trials for every choice trial and found either indifference or a small preference for the suboptimal alternative under these same parameters. Belke and Spetch (1994) used an asymmetrical FE procedure in which choices of the suboptimal alternative that ended in no food were always followed by repeated FE trials of the suboptimal alternative until food was delivered, and they found even stronger preference for the suboptimal alternative. ...
The influence of single option or forced-exposure (FE) trials was studied in the suboptimal choice task. Pigeons chose between an optimal alternative that led to food half of the time and a suboptimal alternative that led to food 20% of the time. Choice of the suboptimal alternative was compared across groups of subjects that received different numbers of FE trials during training. In Experiment 1, subjects received 100% FE trials, 67% FE trials, or only choice trials. Pigeons in the two groups that had FE trials developed extreme preference for the signaled suboptimal alternative over the unsignaled optimal alternative, while pigeons that had no FE trials showed pronounced individual differences. Experiment 2 compared 10% and 90% FE trials. When neither alternative signaled trial outcomes, both groups of subjects strongly preferred the optimal alternative. When the suboptimal alternative provided differential signals, the subjects in the 90% FE group developed strong preference for the suboptimal alternative and subjects in the 10% FE group maintained preference for the optimal alternative. The results of both experiments demonstrate that FE trials can have substantial effects on the development of preference in the suboptimal choice task.
... To test the hypothesis that positive contrast is responsible for the preference for 50% reinforcement alternative over the 100% reinforcement alternative, Zentall et al. (2019) reduced the presumed amount of contrast, by increasing the probability of the signal for reinforcement associated with the suboptimal alternative from 50 to 75%. Thus, instead of a change in the probability of reinforcement from 50% at the time of choice to 100% upon the appearance of the conditioned stimulus (a difference of 50%) there was only a 25% difference. ...
The hypothesis proposed by Macphail (1987) is that differences in intelligent behavior thought to distinguish different species were likely attributed to differences in the context of the tasks being used. Once one corrects for differences in sensory input, motor output, and incentive, it is likely that all vertebrate animals have comparable intellectual abilities. In the present article I suggest a number of tests of this hypothesis with pigeons. In each case, the evidence suggests that either there is evidence for the cognitive behavior, or the pigeons suffer from biases similar to those of humans. Thus, Macphail’s hypothesis offers a challenge to researchers to find the appropriate conditions to bring out in the animal the cognitive ability being tested.
... A series of recent studies by the Beckmann and Zentall laboratories demonstrated that cue presentation is crucial to drive such maladaptive choice behavior. In particular, less predictable [9] or briefer [10] cues associated with suboptimal options lead to greater preference for those. These cues lead to maintenance of suboptimal choice in pigeons, even when reinforcement probabilities were brought as low as 6.25% [11]. ...
In the past several years, there has been an explosion of interest in animal models of risk-based decision-making, a fundamental process associated with gambling disorder. While early work focused on establishing various tasks for assaying decision-making, current studies are determining the (subtle and not-so-subtle) influence of cues in driving risky decisions to better understand problem gambling. In addition, these decision-making paradigms are now being used to investigate comorbid conditions such as substance dependence or brain injury and replicating observations from human patients. These animal models have now developed to a point where therapeutic interventions may be assessed for not just gambling disorder, but also a number of other conditions which engender risky decision-making.
... It can also account for the indifference associated with the choice between the 50% signal for 100% reinforcement and the 100% signal for 100% reinforcement (Smith & Zentall, 2016). However, it is not able to account for the preference for the signal for 100% reinforcement that occurs 50% of the time over the signal for 100% reinforcement that occurs 100% of the time, found with continued training (Zentall et al., 2019). Nor can it account for similar effects reported by others (Belke & Spetch, 1994;Fantino et al., 1979;Kendall, 1974;Mazur, 1996Mazur, , 1998McDevitt et al., 1997;Spetch et al., 1990). ...
When humans engage in commercial gambling, they are making suboptimal choices because the return is typically less than the investment. Pigeons and rats too show suboptimal choice, for example, when pigeons choose between one alternative that 20% of the time provides them with a signal for 100% reinforcement and a second alternative that provides them with a signal for 50% reinforcement. The pigeons’ strong preference for the suboptimal choice is controlled primarily by the value of the signal that predicts reinforcement (a win), rather than how often it occurs (the probability of a win or a loss). A second mechanism, positive contrast between reinforcement expected and obtained, also appears to play a role. Suboptimal choice appears to be related to impulsivity and is correlated with the degree to which reinforcers are discounted by delays. Motivation also plays a role as pigeons on a more restricted diet choose more suboptimally than pigeons on a less restricted diet. In addition, pigeons that spent some time in an enriched environment choose less suboptimally than unenriched pigeons. Research conducted with this animal model of gambling may have implications for the treatment of problem gambling behavior by humans.
... Choosing Option 1 will on occasion deliver the terminal link whose responses pay off 100%, thus providing access to the activity most depressed below the level at which it would otherwise occur. If that outcome is made more probable (say, occurring 1/3 of the time rather than 1/5 of the time), there will be less response deprivation and preference will be less suboptimal (Zentall et al., 2019a); another mini-anomaly. This compounding of the paradox is not explained by simple conditioned reinforcement theory but is by BRT. ...
Two of Timberlake's major contributions, amongst numerous other good notes, are Behavior Regulation Theory (BRT), and Behavior Systems Theory (BST). BRT was a refinement of the Premack Principle. What both got right was that reinforcers are responses, not stimuli. For BRT, they were responses that were occurring below the rate at which they otherwise would given free access to them. BST was a larger ethological framework for our science of behavior. We have always needed it, as it opens an important window on our field. With that window closed, it is easy to stumble over a half-dozen anomalies in the dark, ones that we say humph to, scratch our heads, and then move on. When illuminated by BST, however, such anomalies become keys to a deeper understanding of our subject. This paper reviews numerous anomalies that make sense within the joint framework of BST and BRT, and Dickinson's Dual-Process theory of learned behavior. No longer anomalous in that context, all that is now left to do is test the validity and productivity of this general framework for those many strange cases.
