According to the law of demand, the quantity demanded for a reward is dependent upon the price; as the price of the reward increases, the quantity demanded decreases (1). Conversely, when the price decreases, the quantity demanded increases. This relationship can be observed at any local bar during “happy hour,” a limited time during which the price of beer is discounted. As happy hour draws to a close, patrons quickly consume pints of beer and order “one more round.” The discounted price increases the quantity demanded of beer. The relationship between price and the quantity demanded is formalized using demand curves. Fluctuations in price cause movement along a demand curve, whereas changes in factors such as subjective value or income shift demand. These shifts result in new demand curves (Fig. 1). Understanding the neural mechanisms that participate in this economic tradeoff has implications in neuroscience, economics, and policy making. In PNAS, Schelp et al. (2) demonstrate that dopamine teaching signals, phasic activity that codes for the differences between received and predicted values, cause changes in the behavioral manifestations of economic demand.
Fig. 1.
Prediction error responses influence demand. Demand curves (solid and dashed lines) indicate the relationship between price and quantity demanded. Schelp et al. (2) show that demand can be shifted by modulating dopamine reward prediction error responses. Dopamine activations, positive prediction error responses, decrease price sensitivity and increase demand from baseline (shift from orange to blue curve). Conversely, dopamine suppressions, negative prediction error responses, increase price sensitivity and decrease demand from baseline (shift from orange to gray curve).
Consumers determine demand by calculating the values of rewards they desire, and weighing those values against their income and externally controlled prices. How does the nervous system mediate this transformation from desire to demand? It is likely that neural substrates contributing to this transformation can be found in the reward system. Dopamine neurons respond to rewards and are an ideal candidate to influence consumer demand. These neurons emit action potentials in phasic bursts that code for reward prediction errors, the differences between received and predicted rewards (3, 4), as follows:
Dopaminepredictionerrorresponse=valuereceivedvaluepredicted.
[1]
This response is important for learning the value of rewards and determining the appropriate behavioral responses (5). Positive prediction error responses indicate that a reward was more valuable than predicted, and that behavior preceding the reward should be repeated or invigorated. Conversely, negative prediction error responses indicate that a reward was less valuable than predicted, and that the preceding behavior should be avoided. Updating predictions using this dopamine signal fine-tunes behavior to optimize reward acquisition (6). Previous studies have shown that dopamine signals reflect economic utility, which implicates these neurons in decision making (79).
Monkeys prefer immediate rewards over delayed rewards, and display hyperbolic temporal discounting of reward values. Dopamine responses to delayed rewards follow this hyperbolic decay in reward value (9). Thus, dopamine responses reflect temporally discounted utility. Likewise, uncertainty and risk are ubiquitous features of decision making, and therefore affect utility. Specifically, risk preference determines the shape of utility functions. Dopamine prediction error responses are highly correlated with the shape of utility functions measured under risky conditions (8). Moreover, dopamine responses track dynamically changing utility. Sensory-specific satiety studies, where animals are fed to satiety on a specific reward, measure changes in utility following devaluation. When rats were fed to satiety on a preferred reward, they stopped making choices to get that reward. Dopamine responses to a reward were smaller following devaluation of that reward (10). Thus, dopamine responses reflect subjective value as a function of time and risk preference, as well as satiety. A critical gap in our knowledge is whether dopamine activity reflects the effect of price on utility. The phasic responses of some dopamine neurons (11) and dopamine release in the striatum (12) were diminished by increasing the required effort. However, effort does not have an a priori mapping to cost; therefore, the relationship between economic price and dopamine release remained unclear.
In traditional economic demand curves, price is defined as the dollar amount per unit good. Previous studies have extended the concept of demand curves to behavioral paradigms in experimental animals. In animal studies, price is defined as the number of responses required per reward, and quantity demanded is defined as the amount of consumption. The relationship between required responses and consumption is inverse, just like the relationship between price and quantity demanded (1315). In PNAS, Schelp et al. (2) used this framework to investigate the effect of price on dopamine signaling.
Rats responded to illumination of a cue light by pressing a lever to receive sucrose rewards. Schelp et al. (2) altered the prices of those rewards in two different tasks. The “cost task” increased price by increasing the number of responses required for a fixed reward magnitude. The “reward task” increased price by decreasing the amount of reward delivered for a fixed number of lever presses. To model demand curves, the authors fit the data using an exponential decay model. This model had two parameters, α, the exponential decay, and Qmax, the starting value. Price sensitivity is reflected in α: When price sensitivity is high, α is high, and when price sensitivity is low, α is low. Demand is more robust when α is low, whereas demand is decreased when α is high. Qmax is the maximal consumption when the price is zero, and was estimated from how much sucrose rats consume when given ad libitum access. The authors used this model to quantify the relationship between cost and consumption.
In the cost and reward tasks, rats decreased consumption as cost increased. This behavior demonstrates that the animals accounted for the disutility of higher costs (effort cost + opportunity cost) and behaved in an economically rational way. In the standard cost task, the number of lever presses (i.e., the cost) increased throughout the session. Satiety would also increase throughout the session, and this alignment creates a potential confound between cost and satiety. To disambiguate the effect of cost from the effect of satiety, Schelp et al. (2) used an alternate version of the cost task. In this alternate version, rats started with the highest costs (most lever presses per reward), which progressively decreased across the block. The decreasing costs led to increasing consumption. Thus, the inverse relationship between cost and consumption observed in this study was not explained by satiety. Rather, the disutility associated with cost reduced consumption as predicted by economic theory.
Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra respond to reward and reward-predicting cues according to a reward prediction error algorithm (Eq. 1). These neurons release dopamine in the nucleus accumbens core (NAcc). Accordingly, phasic dopamine responses result in phasic changes in dopamine concentration (dopamine transients) in the NAcc. Fast-scan cyclic voltammetry (16, 17) was used to measure dopamine transients in the NAcc of rats during the cost and reward tasks. Dopamine transients were observed at the onset of the cue and at reward delivery. The magnitudes of the dopamine transients were sensitive to price. Specifically, dopamine responses were larger for lower prices, compared with higher prices, even when the reward itself was identical. This pattern of activation is consistent with a signal that codes for economic value.
Schelp et al. demonstrate that dopamine reward prediction error responses, phasic activity that codes for the difference between received and predicted value, cause changes in economic demand.
To investigate whether dopamine prediction error responses cause changes in economic behavior, Schelp et al. (2) used optogenetic stimulation to augment dopamine release. They measured how augmented dopamine release affects the quantity demanded at various prices. Optogenetics is a technique that uses genetically coded optical actuators [the most common is channelrhodopsin 2 (ChR2)] to perturb neural activity with millisecond time-scale precision (18). The authors infected neurons with a virus that caused ChR2 to be selectively expressed in dopamine neurons. They implanted optical fibers that delivered phasic bursts of light stimulation, and thus were able to activate ChR2-expressing dopamine neurons at the time of the cue or at the time of the reward.
Optogenetic stimulation at the time of the reward enhanced dopamine release and caused the rats to press the lever for higher priced rewards. This behavioral correlate of decreased price sensitivity indicates increased demand, and is consistent with the notion that dopamine activations code for positive prediction errors in subjective value (Eq. 1). The optogenetically enhanced dopamine signal indicated the reward was more valuable than predicted, and increased demand on future trials (Fig. 1, blue arrow). In contrast, optical stimulation at the cue reduced the number of lever presses a rat would perform for a reward. This behavioral correlate of increased price sensitivity indicates decreased demand, and is consistent with the notion that dopamine suppressions code for negative prediction errors in subjective value (Eq. 1). In this case, the predicted value was augmented by optical stimulation. Higher predicted values resulted in dopamine suppressions at reward delivery. Thus, dopamine negative prediction error responses decrease demand (Fig. 1, black arrow).
Dopamine is released in multiple brain areas. To examine whether the dopamine released in the NAcc influenced demand, the authors repeated the optogenetic experiments, but used focal stimulation of dopamine terminals in the NAcc. Although the effect size was smaller, stimulation of dopamine terminals in the NAcc produced qualitatively similar effects as stimulation of dopamine cell bodies in the VTA; that is, increased dopamine release reduced price sensitivity, whereas dopamine suppressions increased price sensitivity. The behavioral shifts in demand curves were robust and reproducible, as shown by Bayesian analysis. Thus, Schelp et al. (2) demonstrate that dopamine reward prediction error responses, phasic activity that codes for the difference between received and predicted value, cause changes in economic demand.
Consumers need mechanisms to adjust their demand in the face of fluctuating prices, changing income, flexible preferences, and other market factors. In this paper, Schelp et al. (2) demonstrate that a well-known neural teaching signal, the dopamine prediction error response, can influence consumer demand. The costs associated with certain rewards affected how dopamine neurons responded. Higher costs evoked smaller dopamine responses, whereas lower costs evoked larger responses. Moreover, dopamine positive prediction error responses cause an increase in demand, whereas dopamine negative prediction error responses cause a decrease in demand. These results are consistent with a neural signal coding for economic utility.
From a neuroscientific perspective, this study addresses an important gap in our understanding of dopamine function. Specifically, this paper addresses how the disutility associated with cost influences dopamine prediction error responses. Previous studies have shown that dopamine responses reflect the utility of rewards (810). Furthermore, dopamine signals are reduced by effort (11, 12, 19). However, it remained unclear whether dopamine signals coded for costs as predicted by economic theory. This study employed an economic framework to measure the price associated with effort and opportunity cost. In doing so, Schelp et al. (2) demonstrate that animals integrate price with reward value as predicted by economic theory. They recorded dopamine signals that scale with the net utility and demonstrated that these signals cause shifts in demand curves. These results provide a biological manifestation of a theoretical economic concept, price, and illustrate how these neural signals adjust fundamental economic behavior.

Acknowledgments

This work was supported by the University of Pittsburgh Brain Institute and by the NIH through NIH Director’s New Innovator Award 1DP2MH113095 (to W.R.S.).

References

1
A Mas-Colell, MD Whinston, JR Green Microeconomic Theory (Oxford Univ Press, New York, 1995).
2
SA Schelp, et al., A transient dopamine signal encodes subjective value and causally influences demand in an economic context. Proc Natl Acad Sci USA 114, E11303–E11312 (2017).
3
W Schultz, Predictive reward signal of dopamine neurons. J Neurophysiol 80, 1–27 (1998).
4
N Eshel, et al., Arithmetic and local circuitry underlying dopamine prediction errors. Nature 525, 243–246 (2015).
5
R Sutton, A Barto Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA, 1998).
6
W Schultz, Reward prediction error. Curr Biol 27, R369–R371 (2017).
7
JA Sugam, JJ Day, RM Wightman, RM Carelli, Phasic nucleus accumbens dopamine encodes risk-based decision-making behavior. Biol Psychiatry 71, 199–205 (2012).
8
WR Stauffer, A Lak, W Schultz, Dopamine reward prediction error responses reflect marginal utility. Curr Biol 24, 2491–2500 (2014).
9
S Kobayashi, W Schultz, Influence of reward delays on responses of dopamine neurons. J Neurosci 28, 7837–7846 (2008).
10
GK Papageorgiou, M Baudonnat, F Cucca, ME Walton, Mesolimbic dopamine encodes prediction errors in a state-dependent manner. Cell Reports 15, 221–228 (2016).
11
B Pasquereau, RS Turner, Limited encoding of effort by dopamine neurons in a cost-benefit trade-off task. J Neurosci 33, 8288–8300 (2013).
12
JJ Day, JL Jones, RM Wightman, RM Carelli, Phasic nucleus accumbens dopamine release encodes effort- and delay-related costs. Biol Psychiatry 68, 306–309 (2010).
13
JH Kagel, RC Battalio, H Rachlin, L Green, Demand curves for animal consumers. Q J Econ 96, 1–15 (1981).
14
JH Kagel, et al., Experimental studies of consumer demand behavior using laboratory animals. Econ Inq 13, 22–38 (1975).
15
SR Hursh, A Silberberg, Economic demand and essential value. Psychol Rev 115, 186–198 (2008).
16
BJ Venton, DJ Michael, RM Wightman, Correlation of local changes in extracellular oxygen and pH that accompany dopaminergic terminal activity in the rat caudate-putamen. J Neurochem 84, 373–381 (2003).
17
BJ Venton, et al., Real-time decoding of dopamine concentration changes in the caudate-putamen during tonic and phasic firing. J Neurochem 87, 1284–1295 (2003).
18
ES Boyden, F Zhang, E Bamberg, G Nagel, K Deisseroth, Millisecond-timescale, genetically targeted optical control of neural activity. Nat Neurosci 8, 1263–1268 (2005).
19
JO Gan, ME Walton, PE Phillips, Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine. Nat Neurosci 13, 25–27 (2010).

Information & Authors

Information

Published in

Go to Proceedings of the National Academy of Sciences
Go to Proceedings of the National Academy of Sciences
Proceedings of the National Academy of Sciences
Vol. 114 | No. 52
December 26, 2017
PubMed: 29233946

Classifications

Submission history

Published online: December 12, 2017
Published in issue: December 26, 2017

Acknowledgments

This work was supported by the University of Pittsburgh Brain Institute and by the NIH through NIH Director’s New Innovator Award 1DP2MH113095 (to W.R.S.).

Notes

See companion article on page E11303.

Authors

Affiliations

Systems Neuroscience Institute, Department of Neurobiology, University of Pittsburgh, Pittsburgh, PA 15213
Systems Neuroscience Institute, Department of Neurobiology, University of Pittsburgh, Pittsburgh, PA 15213

Notes

1
To whom correspondence should be addressed. Email: [email protected].
Author contributions: K.M.R. and W.R.S. wrote the paper.

Competing Interests

The authors declare no conflict of interest.

Metrics & Citations

Metrics

Note: The article usage is presented with a three- to four-day delay and will update daily once available. Due to ths delay, usage data will not appear immediately following publication. Citation information is sourced from Crossref Cited-by service.


Citation statements




Altmetrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.

View Options

View options

PDF format

Download this article as a PDF file

DOWNLOAD PDF

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Personal login Institutional Login

Recommend to a librarian

Recommend PNAS to a Librarian

Purchase options

Purchase this article to get full access to it.

Single Article Purchase

Dopamine prediction error responses update demand
Proceedings of the National Academy of Sciences
  • Vol. 114
  • No. 52
  • pp. 13585-E11336

Media

Figures

Tables

Other

Share

Share

Share article link

Share on social media