Several aspects of the proposal, which aims to expand open access, require serious discussion and, in some cases, a rethink.
Image credit: Dave Cutler (artist).
New Research In
Physical Sciences
Featured Portals
Articles by Topic
Biological Sciences
Featured Portals
Articles by Topic
-
Edited by Dale Purves, Duke University Medical Center, Durham, NC, and approved April 13, 2007 (received for review October 6, 2006)
Abstract
Reinforcement learning models now provide principled guides for a wide range of reward learning experiments in animals and humans. One key learning (error) signal in these models is experiential and reports ongoing temporal differences between expected and experienced reward. However, these same abstract learning models also accommodate the existence of another class of learning signal that takes the form of a fictive error encoding ongoing differences between experienced returns and returns that “could-have-been-experienced” if decisions had been different. These observations suggest the hypothesis that, for all real-world learning tasks, one should expect the presence of both experiential and fictive learning signals. Motivated by this possibility, we used a sequential investment game and fMRI to probe ongoing brain responses to both experiential and fictive learning signals generated throughout the game. Using a large cohort of subjects (n = 54), we report that fictive learning signals strongly predict changes in subjects' investment behavior and correlate with fMRI signals measured in dopaminoceptive structures known to be involved in valuation and choice.
Footnotes
- †To whom correspondence should be addressed. E-mail: tlohrenz{at}hnl.bcm.tmc.edu
-
Author contributions: T.L., K.M., and P.R.M. designed research; T.L. performed research; T.L. and P.R.M. analyzed data; and T.L., K.M., C.F.C., and P.R.M. wrote the paper.
-
Conflict of interest statement: T.L. is Executive Vice President and Director of Research for Computational Management, Inc.
-
This article is a PNAS Direct Submission.
-
This article contains supporting information online at www.pnas.org/cgi/content/full/0608842104/DC1.
- Abbreviations:
- PPC,
- posterior parietal cortex;
- TD,
- temporal difference.
-
Freely available online through the PNAS open access option.
- © 2007 by The National Academy of Sciences of the USA
Neural signature of fictive learning signals in a sequential investment task
Sign up for Article Alerts
Jump to section
You May Also be Interested in
Several large or long-lived animals seem strangely resistant to developing cancer. Elucidating the reasons why could lead to promising cancer-fighting strategies in humans.
Image credit: Shutterstock.com/ronnybas frimages.
PNAS Profile of NAS member and biochemist Hao Wu
A study suggests a putative gene-expression hallmark common to monogamous male vertebrates of some species, namely cichlid fishes, dendrobatid frogs, passeroid songbirds, common voles, and deer mice, and identifies 24 candidate genes potentially associated with monogamy.
Image courtesy of Yusan Yang (University of Pittsburgh, Pittsburgh).
Physical and social well-being in old age are linked to self-assessments of life worth, and a spectrum of behavioral, economic, health, and social variables may influence whether aging individuals believe they are leading meaningful lives.
Image courtesy of Pixabay/MabelAmber.
More Articles of This Classification
Economic Sciences
Biological Sciences
Related Content
Cited by...
- Converging structural and functional connectivity of orbitofrontal, dorsolateral prefrontal, and posterior parietal cortex in the human striatum
- Contrasting temporal difference and opportunity cost reinforcement learning in an empirical money-emergence paradigm
- Interaction of Instrumental and Goal-Directed Learning Modulates Prediction Error Representations in the Ventral Striatum
- Neurocomputational mechanisms of prosocial learning and links to empathy
- Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward
- Dopamine: Context and counterfactuals
- The Good, the Bad, and the Irrelevant: Neural Mechanisms of Learning Real and Hypothetical Rewards and Effort
- Converging Structural and Functional Connectivity of Orbitofrontal, Dorsolateral Prefrontal, and Posterior Parietal Cortex in the Human Striatum
- Belief about nicotine selectively modulates value and reward prediction error signals in smokers
- Differential Modulation of Reinforcement Learning by D2 Dopamine and NMDA Glutamate Receptor Antagonism
- Irrational exuberance and neural crash warning signals during endogenous experimental market bubbles
- Exploiting and exploring the options
- Reward-Related Activity in Ventral Striatum Is Action Contingent and Modulated by Behavioral Relevance
- Hierarchical Learning Induces Two Simultaneous, But Separable, Prediction Errors in Human Basal Ganglia
- Don't Look Back in Anger! Responsiveness to Missed Chances in Successful and Nonsuccessful Aging
- Adaptive coding of reward prediction errors is gated by striatal coupling
- Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning
- Medial prefrontal cortex and striatum mediate the influence of social comparison on the decision process
- Neural Correlates of Forward Planning in a Spatial Decision Task in Humans
- Signals in Human Striatum Are Appropriate for Policy Update Rather than Value Prediction
- A Behavioral and Neural Evaluation of Prospective Decision-Making under Risk
- Neural mechanisms of observational learning
- Brain Hemispheres Selectively Track the Expected Value of Contralateral Options
- Neural Correlates of Value, Risk, and Risk Aversion Contributing to Decision Making under Risk
- Fictive Reward Signals in the Anterior Cingulate Cortex
- Neural correlates of mentalizing-related computations during strategic interactions in humans
- Reinforcement Learning Signals in the Human Striatum Distinguish Learners from Nonlearners during Reward-Based Decision Making