Login | Register

The effect of probability discounting on reward seeking: a three-dimensional perspective


The effect of probability discounting on reward seeking: a three-dimensional perspective

Breton, Yannick-André, Conover, Kent and Shizgal, Peter (2014) The effect of probability discounting on reward seeking: a three-dimensional perspective. Frontiers in Behavioral Neuroscience, 8 (284). pp. 1-13. ISSN 1662-5153

[thumbnail of Publisher's version]
Text (Publisher's version) (application/pdf)
fnbeh-08-00284.pdf - Published Version
Available under License Spectrum Terms of Access.

Official URL: http://dx.doi.org/10.3389/fnbeh.2014.00284


Rats will work for electrical stimulation of the medial forebrain bundle. The rewarding effect arises from the volleys of action potentials fired by the stimulation and subsequent spatio-temporal integration of their post-synpatic impact. The proportion of time allocated to self-stimulation depends on the intensity of the rewarding effect as well as on other key determinants of decision-making, such as subjective opportunity costs and reward probability. We have proposed that a 3D model relating time allocation to the intensity and cost of reward can distinguish manipulations acting prior to the output of the spatio-temporal integrator from those acting at or beyond it. Here, we test this proposition by varying reward probability, a variable that influences the computation of payoff in the 3D model downstream from the output of the integrator. On riskless trials, reward was delivered on every occasion that the rat held down the lever for a cumulative duration called the “price,” whereas on risky trials, reward was delivered with probability 0.75 or 0.50. According to the model, the 3D structure relating time allocation to reward intensity and price is shifted leftward along the price axis by reductions in reward probability; the magnitude of the shift estimates the change in subjective probability. The predictions were borne out: reducing reward probability shifted the 3D structure systematically along the price axis while producing only small, inconsistent displacements along the pulse-frequency axis. The results confirm that the model can accurately distinguish manipulations acting at or beyond the spatio-temporal integrator and strengthen the conclusions of previous studies showing similar shifts following dopaminergic manipulations. Subjective and objective reward probabilities appeared indistinguishable over the range of 0.5 ≤ p ≤ 1.0.

Divisions:Concordia University > Faculty of Arts and Science > Psychology
Concordia University > Research Units > Centre for Studies in Behavioural Neurobiology
Item Type:Article
Authors:Breton, Yannick-André and Conover, Kent and Shizgal, Peter
Journal or Publication:Frontiers in Behavioral Neuroscience
Date:25 August 2014
  • Canadian Institutes of Health Research
  • Fonds de recherche du Québec - Nature et technologies
  • Fonds de recherche du Québec - Santé
  • Concordia Open Access Author Fund
Digital Object Identifier (DOI):10.3389/fnbeh.2014.00284
Keywords:brain-stimulation reward, decision-making, operant conditioning, risk, subjective probability, subjective value
ID Code:978887
Deposited By: Peter Shizgal
Deposited On:09 Oct 2014 15:17
Last Modified:18 Jan 2018 17:47


Arvanitogiannis, A., and Shizgal, P. (2008). The reinforcement mountain: allo- cation of behavior as a function of the rate and intensity of reward- ing brain stimulation. Behav. Neurosci. 122, 1126–1138. doi: 10.1037/ a0012679

Baum, W., and Rachlin, H. (1969). Choice as time allocation. J. Exp. Anal. Behav.
12, 861–874. doi: 10.1901/jeab.1969.12-861

Breton, Y.-A., Marcus, J. C., and Shizgal, P. (2009). Rattus Psychologicus: construc- tion of preferences by self-stimulating rats. Behav. Brain Res. 202, 77–91. doi: 10.1016/j.bbr.2009.03.019

Breton, Y.-A., Mullett, A., Conover, K., and Shizgal, P. (2013). Validation and extension of the reward-mountain model. Front. Behav. Neurosci. 7:125. doi: 10.3389/fnbeh.2013.00125

Charnov, E. L. (1976). Optimal foraging, the marginal value theorem. Theor. Popul.
Biol. 9, 129–136. doi: 10.1016/0040-5809(76)90040-X

Chung, K., and Deisseroth, K. (2013). CLARITY for mapping the nervous system.
Nat. Methods 10, 508–513. doi: 10.1038/nmeth.2481

Deisseroth, K. (2014). Circuit dynamics of adaptive and maladaptive behaviour.
Nature 505, 309–317. doi: 10.1038/nature12982

Gallistel, C. R. (1978). Self-stimulation in the rat: quantitative characteristics of the reward pathway. J. Comp. Physiol. Psychol. 92, 977–998. doi: 10.1037/ h0077513

Gallistel, C. R., and Leon, M. (1991). Measuring the subjective magnitude of brain stimulation reward by titration with rate of reward. Behav. Neurosci. 105, 913–925. doi: 10.1037/0735-7044.105.6.913

Gallistel, C. R., Stellar, J. R., and Bubis, E. (1974). Parametric analysis of brain stimulation reward in the rat: I. The transient process and the memory- containing process. J. Comp. Physiol. Psychol. 87, 848–859. doi: 10.1037/ h0037220

Hau, R., Pleskac, T. J., Kiefer, J., and Hertwig, R. (2008). The description- experience gap in risky choice: the role of sample size and experi- enced probabilities. J. Behav. Decis. Making 21, 493–518. doi: 10.1002/ bdm.598

Hernandez, G., Breton, Y.-A., Conover, K., and Shizgal, P. (2010). At what stage of neural processing does cocaine act to boost pursuit of rewards? PLoS ONE 5:e15081. doi: 10.1371/journal.pone.0015081

Hernandez, G., Trujillo-Pisanty, I., Cossette, M.-P., Conover, K., and Shizgal, P. (2012). Role of dopamine tone in the pursuit of brain stimula- tion reward. J. Neurosci. 32, 11032–11041. doi: 10.1523/JNEUROSCI.1051- 12.2012

Herrnstein, R. (1970). On the law of effect. J. Exp. Anal. Behav. 13, 243–266. doi: 10.1901/jeab.1970.13-243

Herrnstein, R. (1974). Formal properties of the matching law. J. Exp. Anal. Behav. 21, 159–164. doi: 10.1901/jeab.1974.21-159

Kahneman, D., and Tversky, A. (1979). Prospect theory: an analysis of decision under risk. Econometrica 42, 263–292. doi: 10.2307/1914185

Kalenscher, T., and van Wingerden, M. (2011). Why we should use animals to study economic decision making - a perspective. Front. Neurosci. 5:82. doi: 10.3389/fnins.2011.00082

Killeen, P. (1972). The matching law. J. Exp. Anal. Behav. 17, 489–495. doi: 10.1901/jeab.1972.17-489

Miller, H. (1976). Matching-based hedonic scaling in the pigeon. J. Exp. Anal.
Behav. 26, 335–347. doi: 10.1901/jeab.1976.26-335

Prelec, D. (1998). The probability weighting function. Econometrica 66, 497–527. doi: 10.2307/2998573

Simmons, J. M., and Gallistel, C. R. (1994). Saturation of subjective reward magnitude as a function of current and pulse frequency. Behav. Neurosci. 108, 151–160. doi: 10.1037/0735-7044.108.1.151

Solomon, R. B., Conover, K., and Shizgal, P. (2007). “Estimation of subjective opportunity cost in rats working for rewarding brain stimulation: further progress,” in Society for Neuroscience Abstract Viewer (San Diego, CA), 742.8.

Solomon, R. B., Trujillo-Pisanty, I., and Shizgal, P. (2010). “The maximum firing frequency of the neurons subserving brain stimulation reward,” in Society for Neuroscience Abstract Viewer (San Diego, CA), 716.3.

Trujillo-Pisanty, I., Conover, K., and Shizgal, P. (2014). A new view of the effect of dopamine receptor antagonism on operant performance for reward- ing brain stimulation in the rat. Psychopharmacology 231, 1351–1364. doi: 10.1007/s00213-013-3328-x

Trujillo-Pisanty, I., Hernandez, G., Moreau-Debord, I., Cossette, M.-P., Conover, K., Cheer, J. F., et al. (2011). Cannabinoid receptor blockade reduces the opportunity cost at which rats maintain operant performance for rewarding brain stimulation. J. Neurosci. 31, 5426–5435. doi: 10.1523/JNEUROSCI.0079- 11.2011

Yizhar, O., Fenno, L. E., Davidson, T. J., Mogri, M., and Deisseroth,K. (2011). Optogenetics in neural systems. Neuron 71, 9–34. doi: 10.1016/j.neuron.2011.06.004
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.

Repository Staff Only: item control page

Downloads per month over past year

Research related to the current document (at the CORE website)
- Research related to the current document (at the CORE website)
Back to top Back to top