Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward
- Kenneth T. Kishidaa,1,
- Ignacio Saeza,2,
- Terry Lohrenza,
- Mark R. Witcherb,
- Adrian W. Laxtonb,
- Stephen B. Tatterb,
- Jason P. Whitea,
- Thomas L. Ellisb,3,
- Paul E. M. Phillipsc,d, and
- P. Read Montaguea,e,f,1
- aVirginia Tech Carilion Research Institute, Virginia Tech, Roanoke, VA 24016;
- bDepartment of Neurosurgery, Wake Forest Health Sciences, Winston-Salem, NC 27157;
- cDepartment of Psychiatry & Behavioral Sciences, University of Washington, Seattle, WA 98195;
- dDepartment of Pharmacology, University of Washington, Seattle, WA 98195;
- eDepartment of Physics, Virginia Tech, Blacksburg, VA 24060;
- fWellcome Trust Centre for Neuroimaging, University College London, London WC1N 3BG, United Kingdom
-
Edited by Marcus E. Raichle, Washington University in St. Louis, St. Louis, MO, and approved October 23, 2015 (received for review July 13, 2015)
Significance
There is an abundance of circumstantial evidence (primarily work in nonhuman animal models) suggesting that dopamine transients serve as experience-dependent learning signals. This report establishes, to our knowledge, the first direct demonstration that subsecond fluctuations in dopamine concentration in the human striatum combine two distinct prediction error signals: (i) an experience-dependent reward prediction error term and (ii) a counterfactual prediction error term. These data are surprising because there is no prior evidence that fluctuations in dopamine should superpose actual and counterfactual information in humans. The observed compositional encoding of “actual” and “possible” is consistent with how one should “feel” and may be one example of how the human brain translates computations over experience to embodied states of subjective feeling.
Abstract
In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson’s disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson’s disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons.
- dopamine
- reward prediction error
- counterfactual prediction error
- decision-making
- human fast-scan cyclic voltammetry
Footnotes
- ↵1To whom correspondence may be addressed. Email: read{at}vt.edu or kenk{at}vtc.vt.edu.
-
↵2Present address: Helen Wills Neuroscience Institute and Haas School of Business, University of California, Berkeley, CA 94720.
-
↵3Deceased June 30, 2012.
-
Author contributions: K.T.K., T.L., T.L.E., P.E.M.P., and P.R.M. designed research; P.R.M. guided all aspects of this work, including conception of the adaptation of prior rodent microsensor technology for use in humans; T.L. and P.R.M. designed the sequential choice task; M.R.W., A.W.L., S.B.T., and T.L.E. conceived of surgical strategies for safe and effective placement of microsensors for human fast-scan cyclic voltammetry (FSCV) experiments; P.E.M.P. guided microsensor fabrication; I.S. assisted with optimization of microsensor design and engineering of mobile electrochemistry unit; K.T.K., I.S., M.R.W., A.W.L., and S.B.T. performed research; M.R.W., A.W.L., and S.B.T. performed surgical placement of probes; P.E.M.P. guided FSCV experiments; K.T.K. executed FSCV experiments (in vivo and in vitro); I.S. assisted with FSCV experiments (in vivo and in vitro); K.T.K., I.S., J.P.W., and P.E.M.P. contributed new reagents/analytic tools; K.T.K. built and optimized parameters for the extended carbon-fiber microsensors and engineered the integration of mobile electrochemistry unit with game play technology; P.R.M. guided and interpreted signal extraction development and optimization procedures; K.T.K. optimized the signal extraction algorithm using the elastic net; J.P.W. performed temporal alignment of signals collected on electrochemistry unit and integrated game play system (NEMO); K.T.K., I.S., T.L., J.P.W., and P.R.M. analyzed data; P.R.M. guided all analyses; P.R.M. guided and interpreted results from FSCV experiments; K.T.K., I.S., T.L., M.R.W., A.W.L., S.B.T., J.P.W., P.E.M.P., and P.R.M. interpreted results; and K.T.K., T.L., and P.R.M. wrote the paper.
-
The authors declare no conflict of interest.
-
This article is a PNAS Direct Submission.
-
See Commentary on page 22.
-
This article contains supporting information online at www.pnas.org/lookup/suppl/doi:10.1073/pnas.1513619112/-/DCSupplemental.
Freely available online through the PNAS open access option.
