Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward

Kenneth T. Kishida, Ignacio Saez, Terry Lohrenz, Mark R. Witcher, Adrian W. Laxton, Stephen B. Tatter, Jason P. White, Thomas L. Ellis, Paul E.M. Phillips, P. Read Montague

Research output: Contribution to journalArticle

59 Citations (Scopus)

Abstract

In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson's disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson's disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons.

Original languageEnglish (US)
Pages (from-to)200-205
Number of pages6
JournalProceedings of the National Academy of Sciences of the United States of America
Volume113
Issue number1
DOIs
StatePublished - Jan 5 2016
Externally publishedYes

Fingerprint

Reward
Dopamine
Dopaminergic Neurons
Decision Making
Neurotransmitter Agents
Parkinson Disease
Animal Models
Oxygen
Psychiatry
Schizophrenia
Learning
Depression
Brain

Keywords

  • Counterfactual prediction error
  • Decision-making
  • Dopamine
  • Human fast-scan cyclic voltammetry
  • Reward prediction error

ASJC Scopus subject areas

  • General

Cite this

Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward. / Kishida, Kenneth T.; Saez, Ignacio; Lohrenz, Terry; Witcher, Mark R.; Laxton, Adrian W.; Tatter, Stephen B.; White, Jason P.; Ellis, Thomas L.; Phillips, Paul E.M.; Montague, P. Read.

In: Proceedings of the National Academy of Sciences of the United States of America, Vol. 113, No. 1, 05.01.2016, p. 200-205.

Research output: Contribution to journalArticle

Kishida, Kenneth T. ; Saez, Ignacio ; Lohrenz, Terry ; Witcher, Mark R. ; Laxton, Adrian W. ; Tatter, Stephen B. ; White, Jason P. ; Ellis, Thomas L. ; Phillips, Paul E.M. ; Montague, P. Read. / Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward. In: Proceedings of the National Academy of Sciences of the United States of America. 2016 ; Vol. 113, No. 1. pp. 200-205.
@article{5b3d52aaca1f470f8792d14a40d73cea,
title = "Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward",
abstract = "In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson's disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson's disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons.",
keywords = "Counterfactual prediction error, Decision-making, Dopamine, Human fast-scan cyclic voltammetry, Reward prediction error",
author = "Kishida, {Kenneth T.} and Ignacio Saez and Terry Lohrenz and Witcher, {Mark R.} and Laxton, {Adrian W.} and Tatter, {Stephen B.} and White, {Jason P.} and Ellis, {Thomas L.} and Phillips, {Paul E.M.} and Montague, {P. Read}",
year = "2016",
month = "1",
day = "5",
doi = "10.1073/pnas.1513619112",
language = "English (US)",
volume = "113",
pages = "200--205",
journal = "Proceedings of the National Academy of Sciences of the United States of America",
issn = "0027-8424",
number = "1",

}

TY - JOUR

T1 - Subsecond dopamine fluctuations in human striatum encode superposed error signals about actual and counterfactual reward

AU - Kishida, Kenneth T.

AU - Saez, Ignacio

AU - Lohrenz, Terry

AU - Witcher, Mark R.

AU - Laxton, Adrian W.

AU - Tatter, Stephen B.

AU - White, Jason P.

AU - Ellis, Thomas L.

AU - Phillips, Paul E.M.

AU - Montague, P. Read

PY - 2016/1/5

Y1 - 2016/1/5

N2 - In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson's disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson's disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons.

AB - In the mammalian brain, dopamine is a critical neuromodulator whose actions underlie learning, decision-making, and behavioral control. Degeneration of dopamine neurons causes Parkinson's disease, whereas dysregulation of dopamine signaling is believed to contribute to psychiatric conditions such as schizophrenia, addiction, and depression. Experiments in animal models suggest the hypothesis that dopamine release in human striatum encodes reward prediction errors (RPEs) (the difference between actual and expected outcomes) during ongoing decision-making. Blood oxygen level-dependent (BOLD) imaging experiments in humans support the idea that RPEs are tracked in the striatum; however, BOLD measurements cannot be used to infer the action of any one specific neurotransmitter. We monitored dopamine levels with subsecond temporal resolution in humans (n = 17) with Parkinson's disease while they executed a sequential decision-making task. Participants placed bets and experienced monetary gains or losses. Dopamine fluctuations in the striatum fail to encode RPEs, as anticipated by a large body of work in model organisms. Instead, subsecond dopamine fluctuations encode an integration of RPEs with counterfactual prediction errors, the latter defined by how much better or worse the experienced outcome could have been. How dopamine fluctuations combine the actual and counterfactual is unknown. One possibility is that this process is the normal behavior of reward processing dopamine neurons, which previously had not been tested by experiments in animal models. Alternatively, this superposition of error terms may result from an additional yet-to-be-identified subclass of dopamine neurons.

KW - Counterfactual prediction error

KW - Decision-making

KW - Dopamine

KW - Human fast-scan cyclic voltammetry

KW - Reward prediction error

UR - http://www.scopus.com/inward/record.url?scp=84953272555&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84953272555&partnerID=8YFLogxK

U2 - 10.1073/pnas.1513619112

DO - 10.1073/pnas.1513619112

M3 - Article

C2 - 26598677

AN - SCOPUS:84953272555

VL - 113

SP - 200

EP - 205

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 1

ER -