Figure - available from: Frontiers in Human Neuroscience
This content is subject to copyright.
Structure of slot machine game. After introductory instructions and 5 training trials, the player “enters the casino.” He or she is then made to chose a machine, place a bet and pull the lever. As in a standard slot machine, the player watches the wheels spin and is then shown the trial outcome. On 50% of the win trials, the player is allowed to engage in a double-up option. He or she is given 3 seconds to decide whether or not to gamble. On any given trial, the player can also choose to switch the machine or switch between casino visits (not shown). The trial outcomes can be clarified as follows: “true wins” (small and big) were wins, in which the monetary amount won was larger than the original bet. “Fake wins,” were trials in which the monetary amount received was smaller than the original bet placed. “Near-misses” are trials in which the outcome of the trial was a loss, but only the last wheel was different (e.g., AAB), and true losses are trials in which the amount bet was greater than the amount won.
Source publication
Impulsivity plays a key role in decision-making under uncertainty. It is a significant contributor to problem and pathological gambling (PG). Standard assessments of impulsivity by questionnaires, however, have various limitations, partly because impulsivity is a broad, multi-faceted concept. What remains unclear is which of these facets contribute...
Similar publications
Sensation-seeking (SS) is a personality trait that refers to individual differences in motivation for intense and unusual sensory experiences. It describes a facet of human behaviour that has direct relevance for several psychopathologies associated with high social cost. Here, we first review ways of measuring SS behaviour in both humans and anima...
Citations
... acting without deliberation or forethought) play a role in people's reactions to the structural characteristics of gambling such as the near-miss phenomenon. It has been well established that impulsivity plays a key role in decision-making and significantly contributes to problem gambling (Paliwal et al., 2014). This has also been shown in a recent meta-analysis which has found risky gambling behavior to be associated with impulsive decision-making (Ioannidis et al., 2019). ...
... 19%) as an ecologically valid rate for the type of slot machine in our study. This rate is also consistent with other studies (Paliwal et al., 2014). As stated above, participants in the control condition received an identical sequence to the 19% near-miss condition, except that all but 5 of the near-miss trials (2% total) were changed to full loss trials. ...
Both personality factors (e.g. impulsivity) and structural game characteristics impact decision-making on games of chance. We examined the relationship between impulsivity and decision-making on a slot machine task programmed with different near-miss frequencies. Fifty-eight college students entered a simulated casino environment and played a slot machine pre-loaded with 30 credits. Unbeknownst to participants, the slot machine was programmed so that several larger wins occurred early in the sequence, followed by a pattern of diminishing returns that reduced credits to zero on a predetermined trial. Participants were randomly assigned to two conditions, the first with up to 19% of trials set as near misses and the second with only 2% as near misses. After controlling for gender, race, and lifetime gambling frequency, the near-miss condition was found to moderate the relationship between impulsivity and the number of trials played. When there were fewer near misses, impulsivity did not appear to impact decision-making. However, when near misses were frequent, individuals with higher impulsivity persisted longer, even when other characteristics of gameplay remained constant (e.g. bet size, prizes). These findings suggest that certain features of slot machines may capitalize on impulsive gamblers’ vulnerabilities and should be regulated.
... Third, it provides a foundation for RL-style update equations and firmly grounds RL models within the foundations of probability theory. Finally, the hierarchical Gaussian filter has also had considerable success at accounting for a wide variety of empirical phenomena, including impulsivity in healthy individuals (Paliwal, Petzschner, Schmitz, Tittgemeyer, & Stephan, 2014) and Parkinson's patients with deep brain implants (Paliwal et al., 2018), reward-based decision making in schizophrenia (Deserno et al., 2020), social learning (Diaconescu et al., 2017), perceptual learning (Weilnhammer, Stuke, Sterzer, & Schmack, 2018), and sensory learning (Iglesias et al., 2013). ...
The field of mathematical psychology began in the 1950s and includes both psychological theorizing, in which mathematics plays a key role, and applied mathematics motivated by substantive problems in psychology. Central to its success was the publication of the first Handbook of Mathematical Psychology in the 1960s. The psychological sciences have since expanded to include new areas of research, and significant advances have been made both in traditional psychological domains and in the applications of the computational sciences to psychology. Upholding the rigor of the original Handbook, the New Handbook of Mathematical Psychology reflects the current state of the field by exploring the mathematical and computational foundations of new developments over the last half-century. The third volume provides up-to-date, foundational chapters on early vision, psychophysics and scaling, multisensory integration, learning and memory, cognitive control, approximate Bayesian computation, and encoding models in neuroimaging.
... First, researchers have rarely considered these links within healthy populations. Second, previous studies that showed links of impulsivity with reversal performance often provided deterministic switch information despite probabilistic outcomes across a range of values (11,37,43,47). Furthermore, these studies used outcomes of variable amounts, such that participants needed to consider not only uncertainty but also expected outcomes, which could account for some of the findings. ...
Impulsivity is defined as a trait-like tendency to engage in rash actions that are poorly thought out or expressed in an untimely manner. Previous research has found that impulsivity relates to deficits in decision making, in particular when it necessitates executive control or reward outcomes. Reinforcement learning (RL) relies on the ability to integrate reward or punishment outcomes to make good decisions, and has recently been shown to often recruit executive function; as such, it is unsurprising that impulsivity has been studied in the context of RL. However, how impulsivity relates to the mechanisms of RL remains unclear. We aimed to investigate the relationship between impulsivity and learning in a reward-driven learning task with probabilistic feedback and reversal known to recruit executive function. Based on prior literature in clinical populations, we predicted that higher impulsivity would be associated with poorer performance on the task, driven by more frequent switching following unrewarded outcomes. Our results did not support this prediction, but more advanced, trial-history dependent analyses revealed specific effects of impulsivity on switching behavior following consecutive unrewarded trials. Computational modeling captured group-level behavior, but not impulsivity results. Our results support previous findings highlighting the importance of sensitivity to negative outcomes in understanding how impulsivity relates to learning, but indicate that this may stem from more complex strategies than usually considered in computational models of learning. This should be an important target for future research.
... Borderline Personality Disorder (BPD) is characterized by maladaptive decision-making tendencies, such as unstable relationships, self-harm behaviors, and substance use (for review, see Soloff et al., 2000;Dougherty et al., 2004;Rosval et al., 2006;Sebastian et al., 2014). Dimensional approaches propose that the symptoms of BPD reflect instantiations of an underlying predisposition toward impulsivity (i.e., actions without forethought or deliberation; Paliwal et al., 2014), emotion dysregulation, and interpersonal dysfunction (Chapman et al., 2008;Hallquist and Pilkonis, 2012;Scott et al., 2014;Beeney et al., 2018;Hallquist et al., 2018;Allen and Hallquist, 2020), which are transdiagnostic processes that contribute to maladaptive behaviors across several personality pathologies (see Allen and Hallquist, 2020 for a review). Personality and related psychiatric disorders, including BPD, often emerge and intensify during the adolescent period (Johnson et al., 2000a,b;Larsen and Luna, 2018). ...
... Importantly, this paradigm is sensitive to several domains of dysfunction in BPD. First, Impulsivity, given its role in decisionmaking under uncertainty (Dickman, 1993;Zermatten et al., 2005;Paliwal et al., 2014;Chase et al., 2017;Sharma et al., 2017), may play a role in learning and updating the value of available actions (action-outcome contingencies) that change dynamically based on reinforcement history and opponent's behaviors Vickery et al., 2011). Second, emotional dysregulation, which has been shown to exacerbate impulsivity in BPD (Chapman et al., 2008;Sebastian et al., 2013;Neville et al., 2021), may play a role in integrating affective sources of information to update value representations (Paulus et al., 2005). ...
Impulsivity and emotional dysregulation are two core features of borderline personality disorder (BPD), and the neural mechanisms recruited during mixed-strategy interactions overlap with frontolimbic networks that have been implicated in BPD. We investigated strategic choice patterns during the classic two-player game, Matching Pennies, where the most efficient strategy is to choose each option randomly from trial-to-trial to avoid exploitation by one’s opponent. Twenty-seven female adolescents with BPD (mean age: 16 years) and twenty-seven age-matched female controls (mean age: 16 years) participated in an experiment that explored the relationship between strategic choice behavior and impulsivity in both groups and emotional dysregulation in BPD. Relative to controls, BPD participants showed marginally fewer reinforcement learning biases, particularly decreased lose-shift biases, increased variability in reaction times (coefficient of variation; CV), and a greater percentage of anticipatory decisions. A subset of BPD participants with high levels of impulsivity showed higher overall reward rates, and greater modulation of reaction times by outcome, particularly following loss trials, relative to control and BPD participants with lower levels of impulsivity. Additionally, BPD participants with higher levels of emotional dysregulation showed marginally increased reward rate and increased entropy in choice patterns. Together, our preliminary results suggest that impulsivity and emotional dysregulation may contribute to variability in mixed-strategy decision-making in female adolescents with BPD.
... Now compare this to the design of a complex executive control experiment where the subject navigates a decision tree. Decision trees have been used in studies of gambling (Paliwal et al. 2014), foraging (Kolling et al. 2012), and in other executive control tasks (Daw et al. 2005;Gläscher et al. 2010). Here the subject must make a series of decisions, where each decision has specific consequences for the experimental states that follow. ...
Over the past few decades, neuroscience experiments have become increasingly complex and naturalistic. Experimental design has in turn become more challenging, as experiments must conform to an ever-increasing diversity of design constraints. In this article we demonstrate how this design process can be greatly assisted using an optimization tool known as Mixed Integer Linear Programming (MILP). MILP provides a rich framework for incorporating many types of real-world design constraints into a neuroimaging experiment. We introduce the mathematical foundations of MILP, compare MILP to other experimental design techniques, and provide four case studies of how MILP can be used to solve complex experimental design challenges.
... rejected) outcomes, and although this task is wellvalidated, there are other facets of counterfactual thinking that could be relevant to gambling. The consequences of inaction (termed omission bias) or changing one's mind could be relevant to gamblers switching between games, such as individual slot machines (Paliwal, Petzschner, Schmitz, Tittgemeyer, & Stephan, 2014). ...
Counterfactual thinking is a component of human decision-making that entails “if only” thinking about unselected choices and outcomes. It is associated with strong emotional responses of regret (when the obtained outcome is inferior to the counterfactual) and relief (vice versa). Counterfactual thinking may play a role in various cognitive phenomena in disordered gambling, such as the effects of near-misses. This study compared individuals with gambling disorder (n = 46) and healthy controls (n = 25) on a behavioural economic choice task that entailed choosing between two gambles, designed to measure counterfactual thinking. Participants provided affect ratings following both the obtained and the non-obtained outcomes. Choices were analyzed using a computational model that derived parameters reflecting sensitivity to expected value, risk variance, and anticipated regret. In the computational choice model, the group with gambling disorder showed increased sensitivity to anticipated regret, reduced sensitivity to expected value, and increased preference for high risk-variance gambles. On the affect ratings, the group with gambling disorder displayed blunted emotional sensitivity to obtained and counterfactual outcomes. Effect sizes of the group differences were modest. Participants with gambling disorder show wide-ranging alterations in decision-making processes and emotional reactivity to choice outcomes. Altered sensitivity to anticipatory regret in gambling disorder may contribute to the development of gambling-related cognitive distortions, and the influences of gambling marketing.
... Participants gambled on slot machines within a virtual casino (Supplementary Fig. 1) that has previously been used to study impulsive decision-making in healthy controls and patients with Parkinson's disease (Paliwal et al., 2014(Paliwal et al., , 2019). This naturalistic gambling task allowed for impulsive behaviour to be expressed as bet increases, exploratory slot machine switches and 'double or nothing' gambles. ...
These authors contributed equally to this work. Subthalamic deep brain stimulation (STN-DBS) for Parkinson's disease treats motor symptoms and improves quality of life, but can be complicated by adverse neuropsychiatric side-effects, including impulsivity. Several clinically important questions remain unclear: can 'at-risk' patients be identified prior to DBS; do neuropsychiatric symptoms relate to the distribution of the stimulation field; and which brain networks are responsible for the evolution of these symptoms? Using a comprehensive neuropsychiatric battery and a virtual casino to assess impulsive behaviour in a naturalistic fashion, 55 patients with Parkinson's disease (19 females, mean age 62, mean Hoehn and Yahr stage 2.6) were assessed prior to STN-DBS and 3 months postoperatively. Reward evaluation and response inhibition networks were reconstructed with probabilistic tractography using the participant-specific subthalamic volume of activated tissue as a seed. We found that greater connectivity of the stimulation site with these frontostriatal networks was related to greater postoperative impulsiveness and disinhibition as assessed by the neuropsychiatric instruments. Larger bet sizes in the virtual casino postoperatively were associated with greater connectivity of the stimulation site with right and left orbitofrontal cortex, right ventromedial prefrontal cortex and left ventral striatum. For all assessments, the baseline connectivity of reward evaluation and response inhibition networks prior to STN-DBS was not associated with postoperative impulsivity; rather, these relationships were only observed when the stimulation field was incorporated. This suggests that the site and distribution of stimulation is a more important determinant of postoperative neuropsychiatric outcomes than preoperative brain structure and that stimulation acts to mediate impulsivity through differential recruitment of frontostriatal networks. Notably, a distinction could be made amongst participants with clinically-significant, harmful changes in mood and behaviour attributable to DBS, based upon an analysis of connectivity and its relationship with gambling behaviour. Additional analyses suggested that this distinction may be mediated by the differential involvement of fibres connecting ventromedial subthalamic nucleus and orbitofrontal cortex. These findings identify a mechanistic substrate of neuropsychiatric impairment after STN-DBS and suggest that tractography could be used to predict the incidence of adverse neuropsychiatric effects. Clinically, these results highlight the importance of accurate electrode placement and careful stimulation titration in the prevention of neuropsychiatric side-effects after STN-DBS. Abbreviations: DBS = deep brain stimulation; ICB = impulse control behaviour; IFG = inferior frontal gyrus; OFC = orbitofron-tal cortex; SMA = supplementary motor area; STN = subthalamic nucleus; VAT = volume of activated tissue; vmPFC = ventro-medial prefrontal cortex; VTA = ventral tegmental area
... In addition to these classical assessments of impulsivity, participants also gambled on slot machines within a virtual casino, which has been described and validated in healthy controls and individuals with Parkinson's disease (Paliwal et al., 2014(Paliwal et al., , 2019. The motivation for this task was to provide a realistic simulation of impulsive behaviours. ...
See O’Callaghan (doi:10.1093/brain/awz349) for a scientific commentary on this article.
Mosley et al. examine impulsivity and naturalistic gambling behaviours in patients with Parkinson’s disease. They link within-patient differences to the structural connectivity of networks subserving reward evaluation and response inhibition, and reveal pivotal roles for the ventral striatum and subthalamic nucleus within these networks.
... In this analysis, we employed a similar computational framework to that previously reported 19 , applying a hierarchical Bayesian model (the Hierarchical Gaussian Filter, HGF) to behavioural data from 38 participants with PD who played a virtual casino before and after subthalamic DBS. By allowing participants to vary their bet size, switch between slot machines and place 'double or nothing' bets, we could estimate how participants not only inferred the trial-by-trial probability of winning, but also updated higher-order beliefs about the fluctuations (volatility) of a slot machine's winning probability. ...
... By allowing participants to vary their bet size, switch between slot machines and place 'double or nothing' bets, we could estimate how participants not only inferred the trial-by-trial probability of winning, but also updated higher-order beliefs about the fluctuations (volatility) of a slot machine's winning probability. Similar to the rationale outlined in prior work 19 , we believe that a naturalistic paradigm engenders increased behavioural engagement, allowing us to quantify behaviour that has a higher fidelity to 'real world' impulsivity. Additionally, model-based estimates derived from the computational framework may afford us an individual profile of how each participant represented (and responded to) environmental uncertainty. ...
... task. We employed a modified version of an established slot machine gambling paradigm validated in healthy controls 19 . Subjects read an instruction screen and played through 5 training trials, after which they entered a 'virtual' casino, starting with 2000 AUD available to gamble and playing 100 trials (Fig. 1). ...
Subthalamic deep brain stimulation (DBS) for Parkinson’s disease (PD) may modulate chronometric and instrumental aspects of choice behaviour, including motor inhibition, decisional slowing, and value sensitivity. However, it is not well known whether subthalamic DBS affects more complex aspects of decision-making, such as the influence of subjective estimates of uncertainty on choices. In this study, 38 participants with PD played a virtual casino prior to subthalamic DBS (whilst ‘on’ medication) and again, 3-months postoperatively (whilst ‘on’ stimulation). At the group level, there was a small but statistically significant decrease in impulsivity postoperatively, as quantified by the Barratt Impulsiveness Scale (BIS). The gambling behaviour of participants (bet increases, slot machine switches and double or nothing gambles) was associated with this self-reported measure of impulsivity. However, there was a large variance in outcome amongst participants, and we were interested in whether individual differences in subjective estimates of uncertainty (specifically, volatility) were related to differences in pre- and postoperative impulsivity. To examine these individual differences, we fit a computational model (the Hierarchical Gaussian Filter, HGF), to choices made during slot machine game play as well as a simpler reinforcement learning model based on the Rescorla-Wagner formalism. The HGF was superior in accounting for the behaviour of our participants, suggesting that participants incorporated beliefs about environmental uncertainty when updating their beliefs about gambling outcome and translating these beliefs into action. A specific aspect of subjective uncertainty, the participant’s estimate of the tendency of the slot machine’s winning probability to change (volatility), increased subsequent to DBS. Additionally, the decision temperature of the response model decreased post-operatively, implying greater stochasticity in the belief-to-choice mapping of participants. Model parameter estimates were significantly associated with impulsivity; specifically, increased uncertainty was related to increased postoperative impulsivity. Moreover, changes in these parameter estimates were significantly associated with the maximum post-operative change in impulsivity over a six month follow up period. Our findings suggest that impulsivity in PD patients may be influenced by subjective estimates of uncertainty (environmental volatility) and implicate a role for the subthalamic nucleus in the modulation of outcome certainty. Furthermore, our work outlines a possible approach to characterising those persons who become more impulsive after subthalamic DBS, an intervention in which non-motor outcomes can be highly variable.
... We employed a modified version of an established slot machine gambling paradigm validated in healthy controls. 19 Subjects read an instruction screen and played through 5 training trials, after which they entered a 'virtual' casino, starting with 2000 AUD available to gamble and playing 100 trials ( Figure 1). The win-loss likelihood of the slot machines was predetermined and changed at regular intervals. ...
... (ii) Machine-Switch: switching between slot machines (four machines in total) (iii) Casino Switch: cashing out and switching 'virtual' casino days (iv) Double-Up: engaging in a secondary double-or-nothing gamble on certain win trials As in our previous work, 19 these responses, together with trial-wise outcome information (wins/losses), served as the input for our computational models (for a brief summary, see below). Details on the paradigm and computational modelling can be found in prior work 19 and the Supplementary Material. ...
... Based on previous work that examined different computational models of our slot machine paradigm, 19 the perceptual variable used here was simple: a binary variable in which wins were represented by 1 and losses by 0. ...
Subthalamic deep brain stimulation (DBS) for Parkinson's disease (PD) may modulate chronometric and instrumental aspects of choice behaviour, including motor inhibition, decisional slowing, and value sensitivity. However, it is unknown whether subthalamic DBS affects more complex aspects of decision-making, such as estimating the uncertainty around the probability of obtaining rewarding outcomes and the tendency of this probability to change over time. In this study, 38 participants with PD played a slot-machine in a virtual casino prior to subthalamic DBS (whilst 'on' medication) and again, 3-months postoperatively (whilst 'on' stimulation). Gambling behaviour during game play reflected self-reported measures of impulsivity, as quantified by the Barratt Impulsiveness Scale. We fit several computational models, including a hierarchical model of decision-making in the presence of uncertainty (the Hierarchical Gaussian Filter, HGF) and a reinforcement learning model (based on the Rescorla-Wagner formalism), to choices during slot machine play. The HGF was superior in accounting for the behaviour of our participants. Estimates of the perceptual model parameters, which encoded a participant's uncertainty regarding the winning probability of the slot machine and its volatility, were significantly associated with impulsivity. Moreover, preoperative parameter estimates enabled significant out-of-sample predictions of the maximum postoperative change in impulsivity during longitudinal follow up. Our findings suggest that impulsivity in PD patients may be underpinned by uncertainty, and implicate a role for the subthalamic nucleus in the modulation of outcome certainty.
