Lecture 16: Decision Making and Neuroeconomics Flashcards
Decision Making:
simple choice
Decision making is a process that results in the
commitment to a choice among options
What is NEUROECONOMICS?
Neuroeconomics studies decision making by using a
combination of tools from behavioral economics, psychology and neuroscience to avoid the shortcomings that arise from a single-perspective approach.
Expected Value
Probability of Reinforcement
x
Magnitude of Reinforcement
Temporal discounting
compare and contrast neuroeconomics and behavioral economics
- Like behavioral economics, the notion of subjective value is important – the value assigned to an object (or choice option) based on choice behavior as opposed to an inherent property of the object.
– Examples, loss aversion, temporal discounting – expected value vs. subjective value - Unlike behavioral economics, neuroeconomics attempts to determine the complex psychological and brain processes/mechanisms that underlie decisions
Factors in the Decision Process
–Internal/external states
or context
– Long-term goals
– Factors contributing to
valuation (e.g., emotion)
– Learning from past
actions
– Memory
Reward:
a key component of valuation
- A pleasurable event that follows a specific behavior.
- Can be primary (eg, food, sex) or secondary (eg, money).
- The brain uses rewards to learn, choose and prepare/execute goals
The function of reward:
–1) Elicit approach behavior (either through innate mechanisms or learning).
–2) Increase the frequency and intensity of a behavior that leads to a reward (learning).
–3) Induce subjective feelings of pleasure (hedonia) and positive emotional states
Dopamine and Reward
Many rewards lead to release of dopamine
(DA).
Dopamine is released during basic drives (i.e.,
hunger, sex).
DA is released in the rat right before and
during copulation, but not afterwards.
DA is released in the human when
presented with food stimulation in a food-
deprived state.
DA is released when participants are
playing video games for mone
The addicted brain
Dopamine and Reward in the Brain
Value representation in the human brain
Meta analysis of 206 fMRI studies examining neural correlates of subjective value
Two regions show a linear increase with subjective value:
Striatum and Orbitofrontal Cortex
Role of the Striatum
The striatum plays a key role in learning
from reward value
The striatum plays a key role in coding
prediction errors allowing learning from
reward feedback to influence future choices
Reward Prediction Error
Dopamine neurons are active when drop of
liquid is delivered outside any behavioral task.
Earliest predictor of reward signals dopamine
response instead of fully predicted reward
Dopamine neurons will be depressed at the
time of the predicted reward if it fails to
occur.
DA provides an error prediction signal to aid
in goal-directed behavior.
prediction error hypothesis of dopamine
The idea: Dopamine encodes a reward prediction error
prediction error hypothesis of dopamine
p equals probability stimulus predicts delivery of reward.
No response to reward that is fully predicted (p=1),
response shifts to stimulus
Do reward prediction errors drive learning in
humans?
Reward learning in humans
Patients
w/ Parkinson’s disease
Loss of dopamine inputs to the striatum
Patients w/ Parkinson’s disease
FMRI
BOLD signal in striatum correlates parametrically, trial-by-trial with prediction
error (O’Doherty et al. 2003)
Learning reward value
Prediction Error:
A general learning signal across
types of decisions
Example:
Social Reputation Learning
or
learning to trust in a behavioral economics games
The Trust Game
Hyperscanning:
two brains interacting &
learning to trust (or not)
Striatum Response to Reward Feedback During the
Trust Game
Responses to monetary reward
in the striatum diminish and
shift to the presentation of
the trustworthy partner as
reputations are acquired
(and feedback is less
informative) during the
Trust game, i.e., prediction error coding
Orbitofrontal Cortex
- Clearly important in driving optimal
decisions, but what is its role? - Evidence from Patients
- Evidence from fMRI
Phineas Gage
Phineas Gage, (born July 1823, New Hampshire, U.S.—died May 1860, California), American railroad foreman known for having survived a traumatic brain injury caused by an iron rod that shot through his skull and obliterated the greater part of the left frontal lobe of his brain.
Gage’s case is considered to be one of the first examples of scientific evidence indicating that damage to the frontal lobes may alter personality, emotions and social interaction. Prior to this case, the frontal lobes were considered silent structures, without function and unrelated to human behavior.
Neural Systems of Decisions
Orbitofrontal cortex: patient studies
- Base of frontal lobe resting on the orbit of the eyes
Often broken down into 2 regions:
– Ventromedial prefrontal cortex (in red) - This region was damaged on Phineas Gage
– Lateral-orbital prefrontal cortex (in green)
- Precise behavioral function is difficult to define
– “irreverent, indulging at times in grossest
profanity … manifesting but little deference for
his fellows, impatient of restraint or advice when
it conflicts with his desires … devising many plans
of future operation, which are no sooner
arranged than they are abandoned in turn for
others.”
– “Gage is no longer Gage” - Types of behaviors related to human orbitofrontal
cortex may not be clearly analogous in other species - Patients with damage to this region seem to have
difficulties with social and emotional “decision making”
Neural Systems of Decisions:
Orbitofrontal cortex
patient studies- Patient E.V.R.
-orbitofrontal damage
-could generate solutions to problems (e.g., two roommates trying to decide what show to watch)
-could not distinguish which solutions were most likely to be effective
-Damasio hypothesized that emotion was necessary to optimize decision making (“somatic marker hypothesis”)
-showed some evidence of flat affect
Damasio’s Somatic Marker Hypothesis (p. 400, Ward)
emotional information (physiological arousal) guides decision making
-somatic marker = emotional reaction in your body (physiological arousal)
-hypothesis: orbitofrontal region associates situations and somatic changes
-current options evaluated based on likelihood to be rewarding
Initial Support for the Somatic Marker Hypothesis
-Healthy people, or those with
damage to brain regions
outside the orbitofrontal
cortex learn to draw from the
correct pile (pile B; drawing
from pile A will result in net
loss)
-Are they learning by doing
the math, tracking the
probabilities/expected values?
-Or does one pile “feel better”
after a while, and so people
“go with their gut”?
-Healthy people, or those with
damage to brain regions
outside the orbitofrontal cortex
learn to draw from the correct
pile (pile B; drawing from pile
A will result in net loss)
-Patients with orbitofrontal damage do not figure out which pile to draw from. They also do not show increased GSR for risky draws.
-they also show an increased
galvanic skin response when
anticipating a risky draw (from
pile A)
-Suggests that failure to generate somatic marker impairs ability to learn
the task….(what more would be required to convince you?)
Striatal “Reward Prediction Neurons”
- What do dopamine neurons in the striatum respond to?
- Dopamine Response = Reward Occurred – Reward Predicted.
Striatal “Reward Prediction Neurons”
- What do Striatal neurons respond to?
- Experiment: Monkeys is either given an unexpected reward or
trained to associate a light with a reward a few seconds later - Result 1: When a monkey receives a reward without an expectation of
reward, the neuron fires
Result 2: A monkey is trained to associate a flash of light with a reward.
When the predictor (light) is turned on, the dopamine neuron fires.
The dopamine neuron does not respond to the reward. Dopamine neurons show little response when the reward is predicted and received
- Result 3: When the predictor (light) is turned on, the dopamine neuron
fires. When there is no reward, the dopamine neuron drops below
baseline firing.
Dopamine neurons show decreased activity when the reward is expected but not received (i.e.,
“negative prediction error”)
“negative prediction error”
Dopamine neurons show decreased activity when the reward is expected but not received
A study of prediction error in humans
- Experiment setup: participants don’t know how rewarding a specific
shape is. Each time, a shape is followed by nothing or a reward. So, they
learn over time how rewarding something is
- Support from Iowa Gambling Task
- Draw either pile A (riskier pile, both wins and losses are bigger) or pile
B (safer pile, smaller wins and losses)à pile A results in net loss
Healthy subjects eventually learn to draw from pile B
* Healthy subjects also show increased skin conductance response (SCR) when drawing from pile A
In contrast, OFC patients don’t show increased skin conductance response (SCR) when drawing from pile A, and don’t learn to avoid pile A
OFC patients continue to choose pile A (riskier pile), and do not show SCR response
from pile A
* Suggests that failure to generate somatic marker impairs ability to learn from
the task
