Article
|
Open Access
Featured
-
-
Article
| Open AccessHippocampal pattern separation supports reinforcement learning
When learning about rewards and threats in the environment, animals often need to learn the value associated with conjunctions of features, not just individual features. Here, the authors show that the hippocampus forms conjunctive representations that are dissociable from individual feature components.
- Ian C. Ballard
- , Anthony D. Wagner
- & Samuel M. McClure
-
Article
| Open AccessRetrospective model-based inference guides model-free credit assignment
The reinforcement learning literature suggests decisions are based on a model-free system, operating retrospectively, and a model-based system, operating prospectively. Here, the authors show that a model-based retrospective inference of a reward’s cause, guides model-free credit-assignment.
- Rani Moran
- , Mehdi Keramati
- & Raymond J. Dolan
-
Article
| Open AccessNeuronal evidence for good-based economic decisions under variable action costs
Choices between goods often depend on the action costs, but the mechanisms underlying economic decisions under variable action cost are poorly understood. Here, the authors record from neurons in the monkey orbitofrontal cortex and show that decisions under variable action cost were made in a non-spatial representation.
- Xinying Cai
- & Camillo Padoa-Schioppa
-
Article
| Open AccessTime-dependent assessment of stimulus-evoked regional dopamine release
It has proven difficult to measure the release of neurotransmitters, such as dopamine, in the human brain. Here, the authors introduce and validate a new method that infers dopamine release based on minute-by-minute fluctuations of the positron emission tomography (PET) radioligand [11C]raclopride.
- Rachel N. Lippert
- , Anna Lena Cremer
- & Heiko Backes
-
Article
| Open AccessVentral pallidum encodes relative reward value earlier and more robustly than nucleus accumbens
In the ventral basal ganglia circuit, the ventral pallidum (VP) receives major inputs from the nucleus accumbens (NAc) and is involved in reward processing. Here, the authors report that, contrary to the accepted model, signals related to the relative value of reward in VP emerge before NAc and are more robust.
- David Ottenheimer
- , Jocelyn M. Richard
- & Patricia H. Janak
-
Article
| Open AccessReconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex
Working memory (WM) is represented in persistent activity of single neurons as well as a dynamic population code. Here, the authors find that neurons flexibly switch their coding according to current attention while those with stable resting activity maintain WM representations through dynamic activity patterns.
- Sean E. Cavanagh
- , John P. Towers
- & Steven W. Kennerley
-
Article
| Open AccessMultiple timescales of normalized value coding underlie adaptive choice behavior
Previous work has shown that the neural representation of value adapts to the recent history of rewards. Here, the authors report that a computational model based on divisive normalization over multiple timescales can explain changes in value coding driven by changes in the reward statistics.
- Jan Zimmermann
- , Paul W. Glimcher
- & Kenway Louie
-
Article
| Open AccessRole of VTA dopamine neurons and neuroligin 3 in sociability traits related to nonfamiliar conspecific interaction
Individuals with autism spectrum disorder have alteration in social and novelty behaviors. Here, Bellone and colleagues show that chemogenetic inhibition of mouse dopamine neurons in the ventral tegmental area can blunt exploration towards unfamiliar conspecifics, and that these behavioral deficits are recapitulated in mice lacking neuroligin3 gene product.
- Sebastiano Bariselli
- , Hanna Hörnberg
- & Camilla Bellone
-
Article
| Open AccessTranscriptional and physiological adaptations in nucleus accumbens somatostatin interneurons that regulate behavioral responses to cocaine
While making up a small percentage of neurons in the nucleus accumbens, somatostatin interneurons may have important function in dopamine- and addiction-related behavior. Here, Ribeiro and colleagues show that somatostatin interneurons regulate behavioral responses to cocaine with physiological and transcriptomic changes.
- Efrain A. Ribeiro
- , Marine Salery
- & Eric J. Nestler
-
Article
| Open AccessSocial interactions impact on the dopaminergic system and drive individuality
Individual animals differ in behavioral traits, but the mechanisms underlying individuation are unclear. Here, the authors show that mice living in a ‘city’ develop individual behavior differences, associated with changes in dopamine cell firing, that can be reversed on moving them to a different social environment.
- N. Torquet
- , F. Marti
- & P. Faure
-
Article
| Open AccessNicotine aversion is mediated by GABAergic interpeduncular nucleus inputs to laterodorsal tegmentum
Despite its known effects in brain reward centers, nicotine can be aversive in high doses. Here, the authors show that nicotine aversion depends on low-affinity nicotinic acetylcholine receptors expressed on projections from the interpeduncular nucleus to the laterodorsal tegmentum.
- Shannon L. Wolfman
- , Daniel F. Gill
- & Daniel S. McGehee
-
Article
| Open AccessProlonged abstinence from cocaine or morphine disrupts separable valuations during decision conflict
Neuroeconomic theories suggest that conflict during decision, such as exhibited by relapsing drug addicts who continue drug use despite stated wishes not to, might arise from separable processes in decision making. Here the authors test mice in a foraging task designed to separate these processes and find that mice show alterations in separable components of decision conflict following abstinence from cocaine versus morphine.
- Brian M. Sweis
- , A. David Redish
- & Mark J. Thomas
-
Article
| Open AccessDissociable neural mechanisms track evidence accumulation for selection of attention versus action
Decision-making involves parallel information processing regarding what stimulus dimension to pay attention to and what action to take. Here, the authors show that vmPFC tracks the value of the attended attribute while dACC tracks the degree to which it is attended.
- Amitai Shenhav
- , Mark A. Straccia
- & Matthew M. Botvinick
-
Article
| Open AccessAn effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals
Serotonin (5-HT) plays many important roles in reward, punishment, patience and beyond, and optogenetic stimulation of 5-HT neurons has not crisply parsed them. The authors report a novel analysis of a reward-based decision-making experiment, and show that 5-HT stimulation increases the learning rate, but only on a select subset of choices.
- Kiyohito Iigaya
- , Madalena S. Fonseca
- & Peter Dayan
-
Article
| Open AccessReward probability and timing uncertainty alter the effect of dorsal raphe serotonin neurons on patience
Activation of serotonin neurons in the dorsal raphe nucleus promotes patience in waiting for future rewards. Here the authors show that this effect is maximal for high probability reward or high temporal reward uncertainty suggesting that it boosts the prior probability of reward.
- Katsuhiko Miyazaki
- , Kayoko W. Miyazaki
- & Kenji Doya
-
Article
| Open AccessBelief state representation in the dopamine system
Dopamine neurons encode reward prediction errors (RPE) that report the mismatch between expected reward and outcome for a given state. Here the authors report that when there is uncertainty about the current state, RPEs are calculated on the probabilistic representation of the current state or belief state.
- Benedicte M. Babayan
- , Naoshige Uchida
- & Samuel. J. Gershman
-
Article
| Open AccessKnowledge acquisition is governed by striatal prediction errors
Trial and error learning requires the brain to generate expectations and match them to outcomes, yet whether this occurs for semantic learning is unclear. Here, authors show that the brain encodes the degree to which new factual information violates expectations, which in turn determines whether information is encoded in long-term memory.
- Alex Pine
- , Noa Sadeh
- & Avi Mendelsohn
-
Article
| Open AccessIdentity prediction errors in the human midbrain update reward-identity expectations in the orbitofrontal cortex
Responses in the dopaminergic midbrain are known to signal prediction errors for reward value. Here, the authors show that the human midbrain also encodes errors in predicted reward identity, and that these signals update expectations of reward identity in the orbitofrontal cortex.
- James D. Howard
- & Thorsten Kahnt
-
Article
| Open Accessμ-opioid receptor system mediates reward processing in humans
μ-opioid signalling has a known role in the response to various rewarding stimuli, including pleasant foods. Here, Nummenmaa et al. show using PET and fMRI that individual differences in brain μ-opioid receptor density predict the strength of the neural response to highly palatable foods in humans
- Lauri Nummenmaa
- , Tiina Saanijoki
- & Kari Kalliokoski
-
Article
| Open AccessAlcohol exposure disrupts mu opioid receptor-mediated long-term depression at insular cortex inputs to dorsolateral striatum
µ-opioid receptors (MOR) are known to modulate the reward effects of drugs of abuse, and MOR activation induces long-term depression (LTD) at striatal synapses. Here the authors show that alcohol exposure disrupts MOR-induced LTD only at specific cortical inputs to the striatum.
- Braulio Muñoz
- , Brandon M. Fritz
- & Brady K. Atwood
-
Article
| Open AccessActivation of serotonin neurons promotes active persistence in a probabilistic foraging task
Serotonin (5-HT) has been suggested to promote waiting through behavioral inhibition. Here, the authors use an active foraging task and optogenetic activation of 5-HT neurons to show that rather than passivity, these neurons enhance persistence in the face of delay.
- Eran Lottem
- , Dhruba Banerjee
- & Zachary F. Mainen
-
Article
| Open AccessFlexibility in motor timing constrains the topology and dynamics of pattern generator circuits
Human speech and bird song requires the generation of precisely timed motor patterns. The authors show that zebra finches can learn to independently modify the duration of individual song segments and find that synfire chain networks are ideally suited to implement such flexible motor timing.
- Cengiz Pehlevan
- , Farhan Ali
- & Bence P. Ölveczky
-
Article
| Open AccessOpponent control of behavioral reinforcement by inhibitory and excitatory projections from the ventral pallidum
The ventral palladium (VP) is involved in reward and contains mostly GABAergic neurons, but also glutamatergic neurons. Here, the authors characterize glutamatergic VP neurons and show that glutamatergic versus GABAergic projections play roles in avoidance and reinforcement, respectively.
- Lauren Faget
- , Vivien Zell
- & Thomas S. Hnasko
-
Article
| Open AccessA neuronal mechanism underlying decision-making deficits during hyperdopaminergic states
Aberrant increased dopaminergic function results in impaired value-based decision making. Here the authors report pathway-specific effects of VTA activation on distinct aspects of flexible value-based decisions in rats.
- Jeroen P. H. Verharen
- , Johannes W. de Jong
- & Louk J. M. J. Vanderschuren
-
Article
| Open AccessDifferential coding of reward and movement information in the dorsomedial striatal direct and indirect pathways
Classically the direct and indirect pathways of basal ganglia are understood to have opposing roles in movement and reward learning, but recent work suggests a more complicated view. Here the authors further study indirect and direct pathway neurons, in the context of a probabilistic reward task.
- Jung Hwan Shin
- , Dohoung Kim
- & Min Whan Jung
-
Article
| Open AccessChronic alcohol exposure disrupts top-down control over basal ganglia action selection to produce habits
Drug dependence shifts the balance in action selection away from goal-directed to habitual responding. Here, the authors report that chronic passive exposure to alcohol leads to suppression of orbitofrontal cortex inputs to dorsomedial striatum resulting in downregulation of goal-directed behavior.
- Rafael Renteria
- , Emily T. Baltz
- & Christina M. Gremel
-
Article
| Open AccessIncreased striatal activity in adolescence benefits learning
Adolescence is associated with negative behaviors that are related to enhanced reward-related striatal activity, but it is unclear whether this activity could also be beneficial. Here, authors report longitudinal data showing that enhanced striatal activity is related with increased learning ability.
- S. Peters
- & E. A. Crone
-
Article
| Open AccessDistributed neural representation of saliency controlled value and category during anticipation of rewards and punishments
Stimulus category, saliency and value all affect the subjective value estimates that guide our decisions. Here, the authors systematically vary their stimuli along these three dimensions in humans and report category independent encoding of values and saliency in the vmPFC and striatum.
- Zhihao Zhang
- , Jennifer Fanning
- & Ifat Levy
-
Article
| Open AccessInverted activity patterns in ventromedial prefrontal cortex during value-guided decision-making in a less-is-more task
Ventromedial prefrontal cortex in humans shows functional magnetic resonance imaging signals related to the subjective values of choices that are taken during decision-making as well as task-negative signals. Here, the authors report that in macaque ventromedial prefrontal cortex both activity patterns are inverted and lesions of this area disrupt subjective choice evaluation.
- Georgios K. Papageorgiou
- , Jerome Sallet
- & Matthew F. S. Rushworth
-
Article
| Open AccessInhibiting Rho kinase promotes goal-directed decision making and blocks habitual responding for cocaine
Action-outcome learning requires the prelimbic prefrontal cortex. Here the authors report that fasudil, a Rho kinase inhibitor, reduces dendritic spine densities on prelimbic neurons in an activity-dependent manner, stimulating goal-directed actions, and reducing habitual responding for cocaine.
- Andrew M. Swanson
- , Lauren M. DePoy
- & Shannon L. Gourley
-
Article
| Open AccessIngestion of artificial sweeteners leads to caloric frustration memory in Drosophila
While non-caloric artificial sweeteners (NAS) are used as food additives, it’s unclear whether animals perceive NAS as positive or negative percept. Here, Musso and colleagues show in Drosophila that NAS is a negative percept, encoded in a new type of memory.
- Pierre-Yves Musso
- , Aurélie Lampin-Saint-Amaux
- & Thomas Preat
-
Article
| Open AccessShort-term reward experience biases inference despite dissociable neural correlates
Making a good decision often requires the weighing of relative short-term rewards against long-term benefits, yet how the brain does this is not understood. Here, authors show that long-term beliefs are biased by reward experience and that dissociable brain regions facilitate both types of learning.
- Adrian G. Fischer
- , Sacha Bourgeois-Gironde
- & Markus Ullsperger
-
Article
| Open AccessOptimal coding and neuronal adaptation in economic decisions
During economic decisions, offer value cells in orbitofrontal cortex encode the values of offered goods and undergo range adaptation. Here the authors present a theory of optimal coding based on payoff maximization. For linear tuning functions, range adaptation in offer value cells ensures maximal expected payoff.
- Aldo Rustichini
- , Katherine E. Conen
- & Camillo Padoa-Schioppa
-
Article
| Open AccessSpatiotemporal dynamics of information encoding revealed in orbitofrontal high-gamma
High gamma activity (HGA) and local neurons encode similar information, but it’s unclear if this is true when neurons are heterogeneous, as in the orbitofrontal cortex (OFC). Here, Rich & Wallis show that HGA in OFC is closely related to neuron firing, but reveals clearer spatiotemporal dynamics.
- Erin L. Rich
- & Joni D. Wallis
-
Article
| Open AccessPrior preferences beneficially influence social and non-social learning
People often assume that other people share their preferences, but how exactly this bias manifests itself in learning and decision-making is unclear. Here, authors show that a person's own preferences influence learning in both social and non-social situations, and that this bias improves performance.
- Tor Tarantola
- , Dharshan Kumaran
- & Benedetto De Martino
-
Article
| Open AccessReinforcement determines the timing dependence of corticostriatal synaptic plasticity in vivo
Spike timing dependent plasticity (STDP) has been studied extensively in slices but whether such pairings can induce plasticity in vivo is not known. Here the authors report an experimental paradigm that achieves bidirectional corticostriatal STDP in vivo through modulation by behaviourally relevant reinforcement signals, mediated by dopamine and adenosine signaling.
- Simon D. Fisher
- , Paul B. Robertson
- & John N.J. Reynolds
-
Article
| Open AccessCentral insulin modulates food valuation via mesolimbic pathways
The influence of insulin on food preference and the corresponding underlying neural circuits are unknown in humans. Here, the authors show that increasing insulin changes food preference by modulating mesolimbic neural circuits, and that this pattern is changed in insulin-resistant individuals.
- Lena J. Tiedemann
- , Sebastian M. Schmid
- & Stefanie Brassen
-
Article
| Open AccessReactivation of associative structure specific outcome responses during prospective evaluation in reward-based choices
How the brain evaluates options to make a reward-based choice is unclear. Here, authors show that, prior to choice, neural activity patterns to the potential outcomes are reactivated in macaque orbitofrontal cortex, in a way that reflects the unique event sequences leading up to the outcomes.
- Maya Zhe Wang
- & Benjamin Y. Hayden
-
Article
| Open AccessDrug-seeking motivation level in male rats determines offspring susceptibility or resistance to cocaine-seeking behaviour
Drug addiction is partially heritable but the non-genetic inheritance mechanisms are not well understood. The authors show that motivation of male rats in response to cocaine self-administration elicit susceptibility and/or decreased resistance to developing addiction like behaviour in offspring.
- Qiumin Le
- , Biao Yan
- & Lan Ma
-
Article
| Open AccessNeural circuits for long-term water-reward memory processing in thirsty Drosophila
Distinct subsets of dopaminergic PAM neurons have been shown to be involved in short-term and long-term memory for sugar reward. Here the authors report the neural circuits and the cellular and molecular mechanisms for short-term and long-term memory for water reward in thirstyDrosophila.
- Wei-Huan Shyu
- , Tai-Hsiang Chiu
- & Chia-Lin Wu
-
Article
| Open AccessBlunted ventral striatal responses to anticipated rewards foreshadow problematic drug use in novelty-seeking adolescents
Some adolescents seek novelty, but it is unknown whether the brain circuits underlying this behaviour can be used to predict later, problematic behaviour. Here, authors show that diminished ventral striatal and prefrontal activity in response to anticipated rewards at age 14 in these individuals predicts problematic drug use at age 16.
- Christian Büchel
- , Jan Peters
- & Veronika Ziesch
-
Article
| Open AccessDopaminergic dynamics underlying sex-specific cocaine reward
Sex differences in reward processing are at present poorly understood. Calipari and Juarezet al. report oestrous cycle-dependent fluctuations in firing of VTA dopamine neurons that drive alterations in DAT function expressed in terminals in the NAc. These differences underlie enhanced cocaine reward processing during oestrus.
- Erin S. Calipari
- , Barbara Juarez
- & Eric J Nestler
-
Article
| Open AccessVentral tegmental area glutamate neurons co-release GABA and promote positive reinforcement
Ventral tegmental area (VTA) is involved in reward behaviours, but the precise contribution of VTA glutamatergic neurons to this process is not known. Here the authors show that phasic but not sustained optogenetic stimulation of VTA glutamatergic neurons is rewarding and involves co-release of GABA.
- Ji Hoon Yoo
- , Vivien Zell
- & Thomas S. Hnasko
-
Article
| Open AccessPET imaging-guided chemogenetic silencing reveals a critical role of primate rostromedial caudate in reward evaluation
Processing the value of reward is thought to involve the rostromedial caudate (rmCD), but a causal demonstration is lacking in primates. Here the authors use chemogenetics and PET imaging to show that inactivation of rmCD leads to impairments in reward value judgments.
- Yuji Nagai
- , Erika Kikuchi
- & Takafumi Minamimoto
-
Article
| Open AccessPhasic dopamine release in the rat nucleus accumbens predicts approach and avoidance performance
Reward seeking behaviors involve dopamine (DA) release but the circuits underlying avoidance behavior remain comparatively understudied. Here the authors show that phasic increases in DA release in rats are higher for reward and avoidance cues compared with neutral cues and are positively correlated with poor avoidance.
- Ronny N. Gentry
- , Brian Lee
- & Matthew R. Roesch
-
Article
| Open AccessNeurons in the primate dorsal striatum signal the uncertainty of object–reward associations
The dorsal striatum (DS) is a brain region that is thought to aim actions at certain or known rewards. Here, the authors show that an internal-capsule bordering region of the primate DS signals the uncertainty of object-reward associations, suggesting a novel role for the DS in behavior under uncertainty.
- J. Kael White
- & Ilya E. Monosov
-
Article
| Open AccessA dynamic code for economic object valuation in prefrontal cortex neurons
Economic decisions are based on perceived reward value but it is unclear how individual neurons encode value estimates as input for decision mechanisms. Here authors show that dorsolateral prefrontal cortex uses a dynamic value code based on object-specific valuations by single neurons.
- Ken-Ichiro Tsutsui
- , Fabian Grabenhorst
- & Wolfram Schultz
-
Article
| Open AccessObservational learning computations in neurons of the human anterior cingulate cortex
Humans can learn alone or by watching others, strategies which may depend on similar or different neural networks. This study shows that people watching other players in a card game used computations in neurons of their rostral anterior cingulate cortex to learn through observation.
- Michael R. Hill
- , Erie D. Boorman
- & Itzhak Fried
-
Article
| Open AccessNeural processes mediating contextual influences on human choice behaviour
The influence of context on value-based choice is well established but the neural correlates associated with this remain poorly understood. Here the authors perform fMRI in human subjects and find that the hippocampus and ventral tegmental area/substantia nigra are associated with the degree of influence of context on choice behaviour.
- Francesco Rigoli
- , Karl J. Friston
- & Raymond J. Dolan