Login | Register

Reward and Time On the Valuation of Delayed Rewards

Title:

Reward and Time On the Valuation of Delayed Rewards

Abbas, Zarish (2021) Reward and Time On the Valuation of Delayed Rewards. PhD thesis, Concordia University.

[thumbnail of Abbas_PhD_F2021.pdf]
Preview
Text (application/pdf)
Abbas_PhD_F2021.pdf - Accepted Version
Available under License Spectrum Terms of Access.
3MB

Abstract

Time plays an essential role in reward valuation. While animals prefer large rewards to small rewards, the preference reverses if the large reward is delayed too long. This thesis is divided into four chapters that explore this curtailing effect of delay on reward valuation. Chapter 1 introduces delay discounting and neural models of interval timing to provide context for the questions that are asked in this thesis. In Chapter 2, we ask whether repeated treatment with the psychostimulant methylphenidate alters delay discounting in rats in the long run and whether any such effect is contingent on exposure at certain developmental stages. Compared with controls, rats treated with moderate doses of methylphenidate during early adolescence showed a long-term increase in delay discounting, indicating they had become impulsive. This brought up the question of what the drug was affecting: the magnitude of the reward, the perception of delay or how they combine. To gain insight, some of the more rudimentary assumptions about the combination of subjective reward and subjective delay needed to be confirmed. Chapter 3 addresses one of those assumptions: whether the computation of reward magnitude is independent from reward delay. We used intracranial self-stimulation and the reward-mountain model to tackle this question. The reward-mountain model describes how various dimensions of value combine. Through the model, we can computationally derive whether manipulations change the reward sensitivity by affecting the directly stimulated neurons or their integration, or whether they change the output of the integration. In contrast to the common assumption that relative subjective reward intensity and reward delay are computed independently and combined in a simple multiplicative manner, we show that delay affected reward processing at multiple stages during the computation of reward value. When reward was delayed by 2 s and 4 s, not only were later stages of the reward circuitry affected, but a reduction in reward sensitivity was also observed at early stages of processing, at the directly stimulated neurons or their integration. The final chapter discusses the implications of these results, including the possibility that the neural signal for reward magnitude and its delay may be interwoven.

Divisions:Concordia University > Faculty of Arts and Science > Psychology
Item Type:Thesis (PhD)
Authors:Abbas, Zarish
Institution:Concordia University
Degree Name:Ph. D.
Program:Psychology
Date:14 July 2021
Thesis Supervisor(s):Arvanitogiannis, Andreas
Keywords:Reward, Time, Delay discounting, Methylphenidate, Intracranial self-stimulation, Matching law
ID Code:988921
Deposited By: ZARISH ABBAS
Deposited On:29 Nov 2021 16:16
Last Modified:29 Nov 2021 16:16

References:

Abbas, Z., Sweet, A., Hernandez, G., & Arvanitogiannis, A. (2017). Adolescent Exposure to Methylphenidate Increases Impulsive Choice Later in Life. Frontiers in Behavioral Neuroscience, 11, 214. https://doi.org/10.3389/fnbeh.2017.00214
Adriani, W., Canese, R., Podo, F., & Laviola, G. (2007). 1H MRS-detectable metabolic brain changes and reduced impulsive behavior in adult rats exposed to methylphenidate during adolescence. Neurotoxicology and Teratology, 29(1), 116–125. https://doi.org/10.1016/j.ntt.2006.11.010
Ainslie, G. (1975). Specious reward: A behavioral theory of impulsiveness and impulse control. Psychological Bulletin, 82(4), 463–496. https://doi.org/10.1037/h0076860
Ainslie, G. (1992). Picoeconomics: The strategic interaction of successive motivational states within the person (pp. xvi, 440). Cambridge University Press.
Ainslie, G., & Herrnstein, R. J. (1981). Preference reversal and delayed reinforcement. Animal Learning & Behavior, 9(4), 476–482. https://doi.org/10.3758/BF03209777
Alsop, B., & Davison, M. (1986). Preference for multiple versus mixed schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 45(1), 33–45. https://doi.org/10.1901/jeab.1986.45-33
Andersen, S. L., Arvanitogiannis, A., Pliakas, A. M., LeBlanc, C., & Carlezon, W. A. (2002). Altered responsiveness to cocaine in rats exposed to methylphenidate during development. Nature Neuroscience, 5(1), 13–14. https://doi.org/10.1038/nn777
Arvanitogiannis, A., & Shizgal, P. (2008). The reinforcement mountain: Allocation of behavior as a function of the rate and intensity of rewarding brain stimulation. Behavioral Neuroscience, 122(5), 1126–1138. https://doi.org/10.1037/a0012679
Baum, W. M. (1979). Matching, undermatching, and overmatching in studies of choice. Journal of the Experimental Analysis of Behavior, 32(2), 269–281. https://doi.org/10.1901/jeab.1979.32-269
Baum, W. M., & Rachlin, H. C. (1969). Choice as time allocation. Journal of the Experimental Analysis of Behavior, 12(6), 861–874. https://doi.org/10.1901/jeab.1969.12-861
Bavarian, N., Flay, B. R., Ketcham, P. L., & Smit, E. (2015). The Illicit Use of Prescription Stimulants on College Campuses: A Theory-Guided Systematic Review. Health Education & Behavior, 42(6), 719–729. https://doi.org/10.1177/1090198115580576
Benes, F. M., Vincent, S. L., Molloy, R., & Khan, Y. (1996). Increased interaction of dopamine-immunoreactive varicosities with GABA neurons of rat medial prefrontal cortex occurs during the postweanling period. Synapse, 23(4), 237–245. https://doi.org/10.1002/(SICI)1098-2396(199608)23:4<237::AID-SYN1>3.0.CO;2-8
Berg, M. E., & Grace, R. C. (2004). Independence of terminal-link entry rate and immediacy in concurrent chains. Journal of the Experimental Analysis of Behavior, 82(3), 235–251. https://doi.org/10.1901/jeab.2004.82-235
BGM channel. (2015, April 9). Howard Rachlin, “Matching” SQAB. https://www.youtube.com/watch?v=1DtoEv7iLws
Bolaños, C. A., Barrot, M., Berton, O., Wallace-Black, D., & Nestler, E. J. (2003). Methylphenidate treatment during pre- and periadolescence alters behavioral responses to emotional stimuli at adulthood. Biological Psychiatry, 54(12), 1317–1329. https://doi.org/10.1016/S0006-3223(03)00570-5
Borrero, C. S. W., Vollmer, T. R., Borrero, J. C., Bourret, J. C., Sloman, K. N., Samaha, A. L., & Dallery, J. (2010). Concurrent Reinforcement Schedules for Problem Behavior and Appropriate Behavior: Experimental Applications of the Matching Law. Journal of the Experimental Analysis of Behavior, 93(3), 455–469. https://doi.org/10.1901/jeab.2010.93-455
Brandon, C. L., Marinelli, M., & White, F. J. (2003). Adolescent exposure to methylphenidate alters the activity of rat midbrain dopamine neurons. Biological Psychiatry, 54(12), 1338–1344. https://doi.org/10.1016/S0006-3223(03)00787-X
Breton, Y.-A. (2013). Molar and Molecular Models of Performance for Rewarding Brain Stimulation [Phd, Concordia University]. https://spectrum.library.concordia.ca/977979/
Breton, Y.-A., Conover, K., & Shizgal, P. (2014). The effect of probability discounting on reward seeking: A three-dimensional perspective. Frontiers in Behavioral Neuroscience, 8. https://doi.org/10.3389/fnbeh.2014.00284
Breton, Y.-A., Marcus, J. C., & Shizgal, P. (2009). Rattus Psychologicus: Construction of preferences by self-stimulating rats. Behavioural Brain Research, 202(1), 77–91.
Breton, Y.-A., Mullett, A., Conover, K., & Shizgal, P. (2013). Validation and extension of the reward-mountain model. Frontiers in Behavioral Neuroscience, 7. https://doi.org/10.3389/fnbeh.2013.00125
Buhusi, C. V., & Meck, W. H. (2005). What makes us tick? Functional and neural mechanisms of interval timing. Nature Reviews Neuroscience, 6(10), 755–765. https://doi.org/10.1038/nrn1764
Buonomano, D. V. (2017). Your brain is a time machine: The neuroscience of physics and time (First). W.W. Norton & Company, Inc.
Burgos, H., Cofré, C., Hernández, A., Sáez-Briones, P., Agurto, R., Castillo, A., Morales, B., & Zeise, M. L. (2015). Methylphenidate has long-lasting metaplastic effects in the prefrontal cortex of adolescent rats. Behavioural Brain Research, 291, 112–117. https://doi.org/10.1016/j.bbr.2015.05.009
Carlezon, W. A., Mague, S. D., & Andersen, S. L. (2003). Enduring behavioral effects of early exposure to methylphenidate in rats. Biological Psychiatry, 54(12), 1330–1337. https://doi.org/10.1016/j.biopsych.2003.08.020
Catania, A. C. (1963). Concurrent performances: A baseline for the study of reinforcement magnitude. Journal of the Experimental Analysis of Behavior, 6(2), 299–300. https://doi.org/10.1901/jeab.1963.6-299
Cheng, J., Xiong, Z., Duffney, L. J., Wei, J., Liu, A., Liu, S., Chen, G.-J., & Yan, Z. (2014). Methylphenidate Exerts Dose-Dependent Effects on Glutamate Receptors and Behaviors. Biological Psychiatry, 76(12), 953–962. https://doi.org/10.1016/j.biopsych.2014.04.003
Chung, S.-H., & Herrnstein, R. J. (1967). Choice and delay of reinforcement. Journal of the Experimental Analysis of Behavior, 10(1), 67–74. https://doi.org/10.1901/jeab.1967.10-67
Church, R. M. (1984a). Properties of the internal clock. Annals of the New York Academy of Sciences, 423, 566–582. https://doi.org/10.1111/j.1749-6632.1984.tb23459.x
Church, R. M. (1984b). Properties of the internal clock. Annals of the New York Academy of Sciences, 423, 566–582. https://doi.org/10.1111/j.1749-6632.1984.tb23459.x
Church, R. M. (1997). Chapter 2 Timing and temporal search. In C. M. Bradshaw & E. Szabadi (Eds.), Advances in Psychology (Vol. 120, pp. 41–78). North-Holland. https://doi.org/10.1016/S0166-4115(97)80054-4
Church, R. M. (2002). A tribute to John Gibbon. Behavioural Processes, 57(2–3), 261–274. https://doi.org/10.1016/s0376-6357(02)00018-9
Church, R. M. (2003). A concise introduction to scalar timing theory. In Functional and neural mechanisms of interval timing (pp. 3–22). CRC Press/Routledge/Taylor & Francis Group. https://doi.org/10.1201/9780203009574.sec1
Conover, K. L., & Shizgal, P. (19941001a). Competition and summation between rewarding effects of sucrose and lateral hypothalamic stimulation in the rat. Behavioral Neuroscience, 108(3), 537. https://doi.org/10.1037/0735-7044.108.3.537
Conover, K. L., & Shizgal, P. (19941001b). Differential effects of postingestive feedback on the reward value of sucrose and lateral hypothalamic stimulation in rats. Behavioral Neuroscience, 108(3), 559. https://doi.org/10.1037/0735-7044.108.3.559
Conover, K. L., Woodside, B., & Shizgal, P. (1994). Effects of sodium depletion on competition and summation between rewarding effects of salt and lateral hypothalamic stimulation in the rat. Behavioral Neuroscience, 108(3), 549–558. https://doi.org/10.1037//0735-7044.108.3.549
Constantinople, C. M., Piet, A. T., & Brody, C. D. (2019). An Analysis of Decision under Risk in Rats. Current Biology, 29(12), 2066-2074.e5. https://doi.org/10.1016/j.cub.2019.05.013
Cordes, S., & Gallistel, C. R. (2008). Intact Interval Timing in Circadian CLOCK Mutants. Brain Research, 1227, 120–127. https://doi.org/10.1016/j.brainres.2008.06.043
Critchfield, T. S., & Kollins, S. H. (2001). Temporal discounting: Basic research and the analysis of socially important behavior. Journal of Applied Behavior Analysis, 34(1), 101–122. https://doi.org/10.1901/jaba.2001.34-101
Crowley, N. A., Cody, P. A., Davis, M. I., Lovinger, D. M., & Mateo, Y. (2014). Chronic methylphenidate exposure during adolescence reduces striatal synaptic responses to ethanol. European Journal of Neuroscience, 39(4), 548–556. https://doi.org/10.1111/ejn.12426
Davison, M. (1983). Bias and sensitivity to reinforcement in a concurrent-chain schedule. Journal of the Experimental Analysis of Behavior, 40(1), 15–34. https://doi.org/10.1901/jeab.1983.40-15
Davison, M. (1988). Concurrent schedules: Interaction of reinforcer frequency and reinforcer duration. Journal of the Experimental Analysis of Behavior, 49(3), 339–349. https://doi.org/10.1901/jeab.1988.49-339
Davison, M. C. (1976). Preference for fixed-interval schedules: Effects of unequal initial links. Journal of the Experimental Analysis of Behavior, 25(3), 371–376. https://doi.org/10.1901/jeab.1976.25-371
Davison, M., & Hogsden, I. (1984). Concurrent variable-interval schedule performance: Fixed versus mixed reinforcer durations. Journal of the Experimental Analysis of Behavior, 41(2), 169–182. https://doi.org/10.1901/jeab.1984.41-169
Davison, M., & McCarthy, D. (2016). The Matching Law: A Research Review. Routledge.
Devilbiss, D. M., & Berridge, C. W. (2008). Cognition-Enhancing Doses of Methylphenidate Preferentially Increase Prefrontal Cortex Neuronal Responsiveness. Biological Psychiatry, 64(7), 626–635. https://doi.org/10.1016/j.biopsych.2008.04.037
Diesmann, M., Gewaltig, M.-O., & Aertsen, A. (1999). Stable propagation of synchronous spiking in cortical neural networks. Nature, 402(6761), 529–533. https://doi.org/10.1038/990101
Durstewitz, D. (2004). Neural representation of interval time. NeuroReport, 15(5), 745–749.
Elliffe, D., Davison, M., & Landon, J. (2008). Relative Reinforcer Rates and Magnitudes Do Not Control Concurrent Choice Independently. Journal of the Experimental Analysis of Behavior, 90(2), 169–185. https://doi.org/10.1901/jeab.2008.90-169
Epstein, R. (1981). Amount consumed as a function of magazine-cycle duration. Behaviour Analysis Letters, 1(1), 63–66.
Estle, S. J., Green, L., Myerson, J., & Holt, D. D. (2006). Differential effects of amount on temporal and probability discounting of gains and losses. Memory & Cognition, 34(4), 914–928. https://doi.org/10.3758/BF03193437
Evenden, J. L., & Ryan, C. N. (1996). The pharmacology of impulsive behaviour in rats: The effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology, 128(2), 161–170. https://doi.org/10.1007/s002130050121
Fouriezos, G., & Randall, D. (1997). The cost of delaying rewarding brain stimulation. Behavioural Brain Research, 87(1), 111–113. https://doi.org/10.1016/s0166-4328(97)02280-8
Frank, R. A., & Stutz, R. M. (1984). Self-deprivation: A review. Psychological Bulletin, 96(2), 384–393. https://doi.org/10.1037/0033-2909.96.2.384
Gallistel, C. R. (1978). Self-stimulation in the rat: Quantitative characteristics of the reward pathway. Journal of Comparative and Physiological Psychology, 92(6), 977. https://doi.org/10.1037/h0077513
Gallistel, C. R., & King, A. P. (2009). Memory and the computational brain: Why cognitive science will transform neuroscience (pp. xvi, 319). Wiley-Blackwell. https://doi.org/10.1002/9781444310498
Gallistel, C. R., & Leon, M. (1991). Measuring the subjective magnitude of brain stimulation reward by titration with rate of reward. Behavioral Neuroscience, 105(6), 913–925.
Gallistel, C. R., Shizgal, P., & Yeomans, J. S. (1981). A portrait of the substrate for self-stimulation. Psychological Review, 88(3), 228–273. https://doi.org/10.1037/0033-295X.88.3.228
Gibbon, J. (1977). Scalar expectancy theory and Weber’s law in animal timing. Psychological Review, 84(3), 279–325. https://doi.org/10.1037/0033-295X.84.3.279
Gibbon, J., Fairhurst, S., & Goldberg, B. (1997). Chapter 8 Cooperation, conflict and compromise between circadian and interval clocks in pigeons. In C. M. Bradshaw & E. Szabadi (Eds.), Advances in Psychology (Vol. 120, pp. 329–384). North-Holland. https://doi.org/10.1016/S0166-4115(97)80060-X
Giedd, J. N., Blumenthal, J., Jeffries, N. O., Castellanos, F. X., Liu, H., Zijdenbos, A., Paus, T., Evans, A. C., & Rapoport, J. L. (1999). Brain development during childhood and adolescence: A longitudinal MRI study. Nature Neuroscience, 2(10), 861–863. https://doi.org/10.1038/13158
Gogtay, N., Giedd, J. N., Lusk, L., Hayashi, K. M., Greenstein, D., Vaituzis, A. C., Nugent, T. F., Herman, D. H., Clasen, L. S., Toga, A. W., Rapoport, J. L., & Thompson, P. M. (2004). Dynamic mapping of human cortical development during childhood through early adulthood. Proceedings of the National Academy of Sciences, 101(21), 8174–8179.
Grace, R. (1999). The Matching Law And Amount-dependent Exponential Discounting As Accounts Of Self-control Choice. Journal of the Experimental Analysis of Behavior, 71(1), 27–44. https://doi.org/10.1901/jeab.1999.71-27
Grace, R. C. (1995). Independence of reinforcement delay and magnitude in concurrent chains. Journal of the Experimental Analysis of Behavior, 63(3), 255–276. https://doi.org/10.1901/jeab.1995.63-255
Grace, R. C., Bedell, M. A., & Nevin, J. A. (2002). Preference and resistance to change with constant- and variable-duration terminal links: Independence of reinforcement rate and magnitude. Journal of the Experimental Analysis of Behavior, 77(3), 233–255. https://doi.org/10.1901/jeab.2002.77-233
Green, L., Myerson, J., Holt, D. D., Slevin, J. R., & Estle, S. J. (2004). Discounting of Delayed Food Rewards in Pigeons and Rats: Is There a Magnitude Effect? Journal of the Experimental Analysis of Behavior, 81(1), 39–50. https://doi.org/10.1901/jeab.2004.81-39
Green, L., Myerson, J., & McFadden, E. (1997). Rate of temporal discounting decreases with amount of reward. Memory & Cognition, 25, 715–723. https://doi.org/10.3758/BF03211314
Green, L., & Snyderman, M. (1980). Choice between rewards differing in amount and delay: Toward a choice model of self control. Journal of the Experimental Analysis of Behavior, 34(2), 135–147. https://doi.org/10.1901/jeab.1980.34-135
Grossberg, S., & Schmajuk, N. A. (1989). Neural dynamics of adaptive timing and temporal discrimination during associative learning. Neural Networks, 2(2), 79–102. https://doi.org/10.1016/0893-6080(89)90026-9
Hahnloser, R. H. R., Kozhevnikov, A. A., & Fee, M. S. (2002). An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature, 419(6902), 65–70. https://doi.org/10.1038/nature00974
Harper, D. G. C. (1982). Competitive foraging in mallards: “Ideal free’ ducks. Animal Behaviour, 30(2), 575–584. https://doi.org/10.1016/S0003-3472(82)80071-7
Hernandez, G., Breton, Y.-A., Conover, K., & Shizgal, P. (2010). At What Stage of Neural Processing Does Cocaine Act to Boost Pursuit of Rewards? PLOS ONE, 5(11), e15081. https://doi.org/10.1371/journal.pone.0015081
Hernandez, G., Oleson, E. B., Gentry, R. N., Abbas, Z., Bernstein, D. L., Arvanitogiannis, A., & Cheer, J. F. (2014). Endocannabinoids Promote Cocaine-Induced Impulsivity and Its Rapid Dopaminergic Correlates. Biological Psychiatry, 75(6), 487–498. https://doi.org/10.1016/j.biopsych.2013.09.005
Hernandez, G., Trujillo-Pisanty, I., Cossette, M.-P., Conover, K., & Shizgal, P. (2012). Role of Dopamine Tone in the Pursuit of Brain Stimulation Reward. The Journal of Neuroscience, 32(32), 11032–11041. https://doi.org/10.1523/JNEUROSCI.1051-12.2012
Herrnstein, R. J. (1961). Relative and Absolute Strength of Response as a Function of Frequency of Reinforcement1,2. Journal of the Experimental Analysis of Behavior, 4(3), 267–272. https://doi.org/10.1901/jeab.1961.4-267
Herrnstein, R. J., & Loveland, D. H. (1975). Maximizing and matching on concurrent ratio schedules. Journal of the Experimental Analysis of Behavior, 24(1), 107–116. https://doi.org/10.1901/jeab.1975.24-107
Herzog, E. D., Aton, S. J., Numano, R., Sakaki, Y., & Tei, H. (2004). Temporal precision in the mammalian circadian system: A reliable clock from less reliable neurons. Journal of Biological Rhythms, 19(1), 35–46. https://doi.org/10.1177/0748730403260776
Higa, J. J., & Staddon, J. E. R. (1997). Chapter 1 Dynamic models of rapid temporal control in animals. In C. M. Bradshaw & E. Szabadi (Eds.), Advances in Psychology (Vol. 120, pp. 1–40). North-Holland. https://doi.org/10.1016/S0166-4115(97)80053-2
Ito, M., & Asaki, K. (1982). Choice behavior of rats in a concurrent-chains schedule: Amount and delay of reinforcement. Journal of the Experimental Analysis of Behavior, 37(3), 383–392. https://doi.org/10.1901/jeab.1982.37-383
Ivry, R. B., & Schlerf, J. E. (2008). Dedicated and intrinsic models of time perception. Trends in Cognitive Sciences, 12(7), 273–280. https://doi.org/10.1016/j.tics.2008.04.002
Johansson, F., Jirenhed, D.-A., Rasmussen, A., Zucca, R., & Hesslow, G. (2014). Memory trace and timing mechanism localized to cerebellar Purkinje cells. Proceedings of the National Academy of Sciences, 111(41), 14930–14934. https://doi.org/10.1073/pnas.1415371111
Johnson, H. A., Goel, A., & Buonomano, D. V. (2010). Neural dynamics of in vitro cortical networks reflects experienced temporal patterns. Nature Neuroscience, 13(8), 917–919. https://doi.org/10.1038/nn.2579
Karmarkar, U. R., & Buonomano, D. V. (2007). Telling time in the absence of clocks. Neuron, 53(3), 427–438. https://doi.org/10.1016/j.neuron.2007.01.006
Keller, J. V., & Gollub, L. R. (1977). Duration and rate of reinforcement as determinants of concurrent responding. Journal of the Experimental Analysis of Behavior, 28(2), 145–153. https://doi.org/10.1901/jeab.1977.28-145
Kheramin, S., Body, S., Ho, M.-Y., Velázquez-Martinez, D. N., Bradshaw, C. M., Szabadi, E., Deakin, J. F. W., & Anderson, I. M. (2004). Effects of orbital prefrontal cortex dopamine depletion on inter-temporal choice: A quantitative analysis. Psychopharmacology, 175(2), 206–214. https://doi.org/10.1007/s00213-004-1813-y
Killeen, P. R. (1985). Incentive theory: IV. Magnitude of reward. Journal of the Experimental Analysis of Behavior, 43(3), 407–417. https://doi.org/10.1901/jeab.1985.43-407
Kitano, K., Okamoto, H., & Fukai, T. (2003). Time representing cortical activities: Two models inspired by prefrontal persistent activity. Biological Cybernetics, 88(5), 387–394. https://doi.org/10.1007/s00422-002-0390-6
Koob, G. F., & Moal, M. L. (1997). Drug Abuse: Hedonic Homeostatic Dysregulation. Science, 278(5335), 52–58. https://doi.org/10.1126/science.278.5335.52
Kreitzman, L., & Foster, R. (2010). Seasons of Life: The biological rhythms that enable living things to thrive and survive. Profile Books.
Leo, D., Adriani, W., Cavaliere, C., Cirillo, G., Marco, E. M., Romano, E., Porzio, U. D., Papa, M., Perrone-Capano, C., & Laviola, G. (2009). Methylphenidate to adolescent rats drives enduring changes of accumbal Htr7 expression: Implications for impulsive behavior and neuronal morphology. Genes, Brain and Behavior, 8(3), 356–368. https://doi.org/10.1111/j.1601-183X.2009.00486.x
Leon, M., & Gallistel, C. R. (1992). The function relating the subjective magnitude of brain stimulation reward to stimulation strength varies with site of stimulation. Behavioural Brain Research, 52(2), 183–193. https://doi.org/10.1016/s0166-4328(05)80229-3
Leon, M. I., & Gallistel, C. R. (1998). Self-stimulating rats combine subjective reward magnitude and subjective reward rate multiplicatively. Journal of Experimental Psychology. Animal Behavior Processes, 24(3), 265–277. https://doi.org/10.1037//0097-7403.24.3.265
Lewis, P. A., & Miall, R. C. (2003). Distinct systems for automatic and cognitively controlled time measurement: Evidence from neuroimaging. Current Opinion in Neurobiology, 13(2), 250–255. https://doi.org/10.1016/S0959-4388(03)00036-9
Lewis, P. A., Miall, R. C., Daan, S., & Kacelnik, A. (2003). Interval timing in mice does not rely upon the circadian pacemaker. Neuroscience Letters, 348(3), 131–134. https://doi.org/10.1016/s0304-3940(03)00521-4
Logue, A. W., & Chavarro, A. (1987). Effect on choice of absolute and relative values of reinforcer delay, amount, and frequency. Journal of Experimental Psychology: Animal Behavior Processes, 13(3), 280–291. https://doi.org/10.1037/0097-7403.13.3.280
Logue, A. W., Tobin, H., Chelonis, J. J., Wang, R. Y., Geary, N., & Schachter, S. (1992). Cocaine decreases self-control in rats: A preliminary report. Psychopharmacology, 109(1), 245–247. https://doi.org/10.1007/BF02245509
Long, M. A., & Fee, M. S. (2008). Using temperature to analyse temporal dynamics in the songbird motor pathway. Nature, 456(7219), 189–194. https://doi.org/10.1038/nature07448
Low, K. G., & Gendaszek, A. E. (2002). Illicit use of psychostimulants among college students: A preliminary study. Psychology, Health & Medicine, 7(3), 283–287. https://doi.org/10.1080/13548500220139386
Macar, F., Lejeune, H., Bonnet, M., Ferrara, A., Pouthas, V., Vidal, F., & Maquet, P. (2002). Activation of the supplementary motor area and of attentional networks during temporal processing. Experimental Brain Research, 142(4), 475–485. https://doi.org/10.1007/s00221-001-0953-0
Manitt, C., Mimee, A., Eng, C., Pokinko, M., Stroh, T., Cooper, H. M., Kolb, B., & Flores, C. (2011). The Netrin Receptor DCC Is Required in the Pubertal Organization of Mesocortical Dopamine Circuitry. Journal of Neuroscience, 31(23), 8381–8394.
Matell, M. S., & Meck, W. H. (2004). Cortico-striatal circuits and interval timing: Coincidence detection of oscillatory processes. Brain Research. Cognitive Brain Research, 21(2), 139–170. https://doi.org/10.1016/j.cogbrainres.2004.06.012
Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In The effect of delay and of intervening events on reinforcement value (pp. 55–73). Lawrence Erlbaum Associates, Inc.
Mazur, J. E., Stellar, J. R., & Waraczynski, M. (1987). Self-control choice with electrical stimulation of the brain as a reinforcer. Behavioural Processes, 15(2–3), 143–153. https://doi.org/10.1016/0376-6357(87)90003-9
McDowell, J. J. (2005). On The Classic And Modern Theories Of Matching. Journal of the Experimental Analysis of Behavior, 84(1), 111–127. https://doi.org/10.1901/jeab.2005.59-04
McLean, A. P., & Blampied, N. M. (2001). Sensitivity to relative reinforcer rate in concurrent schedules: Independence from relative and absolute reinforcer duration. Journal of the Experimental Analysis of Behavior, 75(1), 25–42. https://doi.org/10.1901/jeab.2001.75-25
Medina, J. F., Garcia, K. S., Nores, W. L., Taylor, N. M., & Mauk, M. D. (2000). Timing mechanisms in the cerebellum: Testing predictions of a large-scale computer simulation. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 20(14), 5516–5525.
Michon, J. A. (1985). The Compleat Time Experiencer. In J. A. Michon & J. L. Jackson (Eds.), Time, Mind, and Behavior (pp. 20–52). Springer. https://doi.org/10.1007/978-3-642-70491-8_2
Moll, G. H., Hause, S., Rüther, E., Rothenberger, A., & Huether, G. (2001). Early Methylphenidate Administration to Young Rats Causes a Persistent Reduction in the Density of Striatal Dopamine Transporters. Journal of Child and Adolescent Psychopharmacology, 11(1), 15–24. https://doi.org/10.1089/104454601750143366
Moore, J. W. (Ed.). (2002). A Neuroscientist’s Guide to Classical Conditioning. Springer-Verlag. https://doi.org/10.1007/978-1-4419-8558-3
Naneix, F., Marchand, A. R., Di Scala, G., Pape, J.-R., & Coutureau, E. (2012). Parallel Maturation of Goal-Directed Behavior and Dopaminergic Systems during Adolescence. Journal of Neuroscience, 32(46), 16223–16232.
Navarick, D. J., & Fantino, E. (1976). Self-control and general models of choice. Journal of Experimental Psychology: Animal Behavior Processes, 2(1), 75–87. https://doi.org/10.1037/0097-7403.2.1.75
Neuringer, A. J. (1967). Effects of reinforcement magnitude on choice and rate of responding. Journal of the Experimental Analysis of Behavior, 10(5), 417–424. https://doi.org/10.1901/jeab.1967.10-417
Nieuwenhuys, R., Geeraedts, L. M. G., & Veening, J. G. (1982). The medial forebrain bundle of the rat. I. General introduction. Journal of Comparative Neurology, 206(1), 49–81. https://doi.org/10.1002/cne.902060106
Odum, A. L., & Baumann, A. A. L. (2010). Delay discounting: State and trait variable. In Impulsivity: The behavioral and neurological science of discounting (pp. 39–65). American Psychological Association. https://doi.org/10.1037/12069-002
Olds, J. (20060327). Satiation effects in self-stimulation of the brain. Journal of Comparative and Physiological Psychology, 51(6), 675. https://doi.org/10.1037/h0039616
Ong, E. L., & White, K. G. (2004). Amount-dependent temporal discounting? Behavioural Processes, 66(3), 201–212. https://doi.org/10.1016/j.beproc.2004.03.005
Paine, T. A., Dringenberg, H. C., & Olmstead, M. C. (2003). Effects of chronic cocaine on impulsivity: Relation to cortical serotonin mechanisms. Behavioural Brain Research, 147(1–2), 135–147. https://doi.org/10.1016/s0166-4328(03)00156-6
Papachristos, E. B., Jacobs, E. H., & Elgersma, Y. (2011). Interval timing is intact in arrhythmic Cry1/Cry2-deficient mice. Journal of Biological Rhythms, 26(4), 305–313. https://doi.org/10.1177/0748730411410026
Pardey, M. C., Kumar, N. N., Goodchild, A. K., Clemens, K. J., Homewood, J., & Cornish, J. L. (2012). Long-Term Effects of Chronic Oral Ritalin Administration on Cognitive and Neural Development in Adolescent Wistar Kyoto Rats. Brain Sciences, 2(3), 375–404. https://doi.org/10.3390/brainsci2030375
Patke, A., Young, M. W., & Axelrod, S. (2020). Molecular mechanisms and physiological importance of circadian rhythms. Nature Reviews Molecular Cell Biology, 21(2), 67–84. https://doi.org/10.1038/s41580-019-0179-2
Pietras, C. J., Cherek, D. R., Lane, S. D., Tcheremissine, O. V., & Steinberg, J. L. (2003). Effects of methylphenidate on impulsive choice in adult humans. Psychopharmacology, 170(4), 390–398. https://doi.org/10.1007/s00213-003-1547-2
Pine, A., Shiner, T., Seymour, B., & Dolan, R. J. (2010). Dopamine, Time, and Impulsivity in Humans. Journal of Neuroscience, 30(26), 8888–8896.
Pitts, R. C., & McKinney, A. P. (2005). Effects of methylphenidate and morphine on delay-discount functions obtained within sessions. Journal of the Experimental Analysis of Behavior, 83(3), 297–314. https://doi.org/10.1901/jeab.2005.47-04
Plaud, J. J. (1992). The prediction and control of behavior revisited: A review of the matching law. Journal of Behavior Therapy and Experimental Psychiatry, 23(1), 25–31. https://doi.org/10.1016/0005-7916(92)90021-A
Rachlin, H. (2006). Notes on Discounting. Journal of the Experimental Analysis of Behavior, 85(3), 425–435. https://doi.org/10.1901/jeab.2006.85-05
Rachlin, H., & Baum, W. M. (1972). Effects of Alternative Reinforcement: Does the Source Matter?1. Journal of the Experimental Analysis of Behavior, 18(2), 231–241. https://doi.org/10.1901/jeab.1972.18-231
Rachlin, H., & Green, L. (1972). Commitment, Choice and Self-Control1. Journal of the Experimental Analysis of Behavior, 17(1), 15–22. https://doi.org/10.1901/jeab.1972.17-15
Raible, F., Takekata, H., & Tessmar-Raible, K. (2017). An Overview of Monthly Rhythms and Clocks. Frontiers in Neurology, 8. https://doi.org/10.3389/fneur.2017.00189
Raineri, A., & Rachlin, H. (1993). The effect of temporal constraints on the value of money and other commodities. Journal of Behavioral Decision Making, 6(2), 77–94. https://doi.org/10.1002/bdm.3960060202
Rammsayer, T. H. (1999). Neuropharmacological evidence for different timing mechanisms in humans. The Quarterly Journal of Experimental Psychology. B, Comparative and Physiological Psychology, 52(3), 273–286. https://doi.org/10.1080/713932708
Rammsayer, T. H., & Lima, S. D. (1991). Duration discrimination of filled and empty auditory intervals: Cognitive and perceptual factors. Perception & Psychophysics, 50(6), 565–574. https://doi.org/10.3758/bf03207541
Reed, D. D., & Kaplan, B. A. (2011). The Matching Law: A Tutorial for Practitioners. Behavior Analysis in Practice, 4(2), 15–24. https://doi.org/10.1007/BF03391780
Reppert, S. M., & Weaver, D. R. (2002). Coordination of circadian timing in mammals. Nature, 418(6901), 935–941. https://doi.org/10.1038/nature00965
Reynolds, L. M., Makowski, C. S., Yogendran, S. V., Kiessling, S., Cermakian, N., & Flores, C. (2015). Amphetamine in Adolescence Disrupts the Development of Medial Prefrontal Cortex Dopamine Connectivity in a dcc -Dependent Manner. Neuropsychopharmacology, 40(5), 1101–1112. https://doi.org/10.1038/npp.2014.287
Richards, J. B., Mitchell, S. H., de Wit, H., & Seiden, L. S. (1997). Determination of discount functions in rats with an adjusting-amount procedure. Journal of the Experimental Analysis of Behavior, 67(3), 353–366. https://doi.org/10.1901/jeab.1997.67-353
Rider, D. P. (1983). Choice for aperiodic versus periodic ratio schedules: A comparison of concurrent and concurrent-chain procedures. Journal of the Experimental Analysis of Behavior, 40(3), 225–237. https://doi.org/10.1901/jeab.1983.40-225
Rodriguez, M. L., & Logue, A. W. (1986). Independence of the amount and delay ratios in the generalized matching law. Animal Learning & Behavior, 14(1), 29–37. https://doi.org/10.3758/BF03200034
Schibler, U., & Sassone-Corsi, P. (2002). A web of circadian pacemakers. Cell, 111(7), 919–922. https://doi.org/10.1016/s0092-8674(02)01225-4
Shiels, K., Hawk, L. W., Reynolds, B., Mazzullo, R., Rhodes, J., Pelham, W. E., Waxmonsky, J. G., & Gangloff, B. P. (2009). The Effects of Methylphenidate on Discounting of Delayed Rewards in ADHD. Experimental and Clinical Psychopharmacology, 17(5), 291–301. https://doi.org/10.1037/a0017259
Shizgal, P. (1997). Neural basis of utility estimation. Current Opinion in Neurobiology, 7(2), 198–208. https://doi.org/10.1016/S0959-4388(97)80008-6
Shuler, M. G., & Bear, M. F. (2006). Reward Timing in the Primary Visual Cortex. Science, 311(5767), 1606–1609. https://doi.org/10.1126/science.1123513
Shull, R. L., & Pliskoff, S. S. (1967). Changeover delay and concurrent schedules: Some effects on relative performance measures. Journal of the Experimental Analysis of Behavior, 10(6), 517–527. https://doi.org/10.1901/jeab.1967.10-517
Simmons, J. M., & Gallistel, C. R. (1994). Saturation of subjective reward magnitude as a function of current and pulse frequency. Behavioral Neuroscience, 108(1), 151–160. https://doi.org/10.1037//0735-7044.108.1.151
Simon, N. W., Mendez, I. A., & Setlow, B. (2007). Cocaine exposure causes long-term increases in impulsive choice. Behavioral Neuroscience, 121(3), 543–549. https://doi.org/10.1037/0735-7044.121.3.543
Snyderman, M. (1983). Delay and amount of reward in a concurrent chain. Journal of the Experimental Analysis of Behavior, 39(3), 437–447. https://doi.org/10.1901/jeab.1983.39-437
Solomon, R. B., Trujillo-Pisanty, I., Conover, K., & Shizgal, P. (2015). Psychophysical inference of frequency-following fidelity in the neural substrate for brain stimulation reward. Behavioural Brain Research, 292, 327–341. https://doi.org/10.1016/j.bbr.2015.06.008
Somkuwar, S. S., Jordan, C. J., Kantak, K. M., & Dwoskin, L. P. (2013). Adolescent Atomoxetine Treatment in a Rodent Model of ADHD: Effects on Cocaine Self-Administration and Dopamine Transporters in Frontostriatal Regions. Neuropsychopharmacology, 38(13), 2588–2597. https://doi.org/10.1038/npp.2013.163
Sonnenschein, B., Conover, K., & Shizgal, P. (2003). Growth of brain stimulation reward as a function of duration and stimulation strength. Behavioral Neuroscience, 117(5), 978–994. https://doi.org/10.1037/0735-7044.117.5.978
Spear, L. P. (2000). The adolescent brain and age-related behavioral manifestations. Neuroscience & Biobehavioral Reviews, 24(4), 417–463. https://doi.org/10.1016/S0149-7634(00)00014-2
Squires, N., & Fantino, E. (1971). A model for choice in simple concurrent and concurrent-chains schedules. Journal of the Experimental Analysis of Behavior, 15(1), 27–38. https://doi.org/10.1901/jeab.1971.15-27
Staddon, J. E., & Higa, J. J. (1999). Time and memory: Towards a pacemaker-free theory of interval timing. Journal of the Experimental Analysis of Behavior, 71(2), 215–251. https://doi.org/10.1901/jeab.1999.71-215
Thornquist, S. C., Langer, K., Zhang, S. X., Rogulja, D., & Crickmore, M. A. (2020). CaMKII Measures the Passage of Time to Coordinate Behavior and Motivational State. Neuron, 105(2), 334-345.e9. https://doi.org/10.1016/j.neuron.2019.10.018
Thornquist, S. C., Pitsch, M. J., Auth, C. S., & Crickmore, M. A. (2021). Biochemical evidence accumulates across neurons to drive a network-level eruption. Molecular Cell, 81(4), 675-690.e8. https://doi.org/10.1016/j.molcel.2020.12.029
Tieu, K. H., Keidel, A. L., McGann, J. P., Faulkner, B., & Brown, T. H. (1999). Perirhinal-amygdala circuit-level computational model of temporal encoding in fear conditioning. Psychobiology, 27(1), 1–25. https://doi.org/10.3758/BF03332095
Todorov, J. C. (1973). Interaction of frequency and magnitude of reinforcement on concurrent performances. Journal of the Experimental Analysis of Behavior, 19(3), 451–458. https://doi.org/10.1901/jeab.1973.19-451
Trujillo-Pisanty, I., Conover, K., & Shizgal, P. (2013). A new view of the effect of dopamine receptor antagonism on operant performance for rewarding brain stimulation in the rat. Psychopharmacology. https://doi.org/10.1007/s00213-013-3328-x
Trujillo-Pisanty, I., Conover, K., Solis, P., Palacios, D., & Shizgal, P. (2020). Dopamine neurons do not constitute an obligatory stage in the final common path for the evaluation and pursuit of brain stimulation reward. PloS One, 15(6), e0226722. https://doi.org/10.1371/journal.pone.0226722
Trujillo-Pisanty, I., Hernandez, G., Moreau-Debord, I., Cossette, M.-P., Conover, K., Cheer, J. F., & Shizgal, P. (2011). Cannabinoid receptor blockade reduces the opportunity cost at which rats maintain operant performance for rewarding brain stimulation. The Journal of Neuroscience: The Official Journal of the Society for Neuroscience, 31(14), 5426–5435. https://doi.org/10.1523/JNEUROSCI.0079-11.2011
Urban, K. R., & Gao, W.-J. (2015). Evolution of the Study of Methylphenidate and Its Actions on the Adult Versus Juvenile Brain. Journal of Attention Disorders, 19(7), 603–619. https://doi.org/10.1177/1087054712455504
van der Marel, K., Klomp, A., Meerhoff, G. F., Schipper, P., Lucassen, P. J., Homberg, J. R., Dijkhuizen, R. M., & Reneman, L. (2014). Long-Term Oral Methylphenidate Treatment in Adolescent and Adult Rats: Differential Effects on Brain Morphology and Function. Neuropsychopharmacology, 39(2), 263–273. https://doi.org/10.1038/npp.2013.169
van Gaalen, M. M., van Koten, R., Schoffelmeer, A. N. M., & Vanderschuren, L. J. M. J. (2006). Critical involvement of dopaminergic neurotransmission in impulsive decision making. Biological Psychiatry, 60(1), 66–73. https://doi.org/10.1016/j.biopsych.2005.06.005
van Rijn, H., Gu, B.-M., & Meck, W. H. (2014). Dedicated clock/timing-circuit theories of time perception and timed performance. Advances in Experimental Medicine and Biology, 829, 75–99. https://doi.org/10.1007/978-1-4939-1782-2_5
Vanderveldt, A., Oliveira, L., & Green, L. (2016). Delay Discounting: Pigeon, Rat, Human – Does it Matter? Journal of Experimental Psychology. Animal Learning and Cognition, 42(2), 141–162. https://doi.org/10.1037/xan0000097
Vendruscolo, L. F., Izídio, G. S., Takahashi, R. N., & Ramos, A. (2008). Chronic methylphenidate treatment during adolescence increases anxiety-related behaviors and ethanol drinking in adult spontaneously hypertensive rats. Behavioural Pharmacology, 19(1), 21–27. https://doi.org/10.1097/FBP.0b013e3282f3cfbe
Ward, R. D., Gallistel, C. R., & Balsam, P. D. (2013). It’s The Information! Behavioural Processes, 95, 3–7. https://doi.org/10.1016/j.beproc.2013.01.005
Welsh, D. K., Engel, E. M., Richardson, G. S., Dement, W. C., & Engle, E. M. (1986). Precision of circadian wake and activity onset timing in the mouse. Journal of Comparative Physiology. A, Sensory, Neural, and Behavioral Physiology, 158(6), 827–834. https://doi.org/10.1007/BF01324824
White, K. G., & Pipe, M.-E. (1987). Sensitivity to reinforcer duration in a self-control procedure. Journal of the Experimental Analysis of Behavior, 48(2), 235–249. https://doi.org/10.1901/jeab.1987.48-235
Winstanley, C. (2010). The neural and neurochemical basis of delay discounting. In Madden, G.J & Bickel, W.K., Impulsivity: The behavioral and neurological science of discounting (pp. 95–121). American Psychological Association. https://content.apa.org/record/2009-18177-004
Wit, H., & Mitchell, S. (2010). Drug effects on delay discounting. Impulsivity: The Behavioral and Neurological Science of Discounting, 213–241. https://doi.org/10.1037/12069-008
Yamazaki, T., & Tanaka, S. (2005). Neural modeling of an internal clock. Neural Computation, 17(5), 1032–1058. https://doi.org/10.1162/0899766053491850
Zeeb, F. D., Floresco, S. B., & Winstanley, C. A. (2010). Contributions of the orbitofrontal cortex to impulsive choice: Interactions with basal levels of impulsivity, dopamine signalling, and reward-related cues. Psychopharmacology, 211(1), 87–98. https://doi.org/10.1007/s00213-010-1871-2
Zosel, A., Bartelson, B. B., Bailey, E., Lowenstein, S., & Dart, R. (2013). Characterization of adolescent prescription drug abuse and misuse using the Researched Abuse Diversion and Addiction-related Surveillance (RADARS(®)) System. Journal of the American Academy of Child and Adolescent Psychiatry, 52(2), 196-204.e2. https://doi.org/10.1016/j.jaac.2012.11.014
All items in Spectrum are protected by copyright, with all rights reserved. The use of items is governed by Spectrum's terms of access.

Repository Staff Only: item control page

Downloads per month over past year

Research related to the current document (at the CORE website)
- Research related to the current document (at the CORE website)
Back to top Back to top