Lecture 16: Decision Making and Neuroeconomics Flashcards

1
Q

Decision Making:
simple choice

A

Decision making is a process that results in the
commitment to a choice among options

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is NEUROECONOMICS?

A

Neuroeconomics studies decision making by using a
combination of tools from behavioral economics, psychology and neuroscience to avoid the shortcomings that arise from a single-perspective approach.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Expected Value

A

Probability of Reinforcement
x
Magnitude of Reinforcement

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Temporal discounting

A
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

compare and contrast neuroeconomics and behavioral economics

A
  • Like behavioral economics, the notion of subjective value is important – the value assigned to an object (or choice option) based on choice behavior as opposed to an inherent property of the object.
    – Examples, loss aversion, temporal discounting – expected value vs. subjective value
  • Unlike behavioral economics, neuroeconomics attempts to determine the complex psychological and brain processes/mechanisms that underlie decisions
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Factors in the Decision Process

A

–Internal/external states
or context
– Long-term goals
– Factors contributing to
valuation (e.g., emotion)
– Learning from past
actions
– Memory

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Reward:
a key component of valuation

A
  • A pleasurable event that follows a specific behavior.
  • Can be primary (eg, food, sex) or secondary (eg, money).
  • The brain uses rewards to learn, choose and prepare/execute goals
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

The function of reward:

A

–1) Elicit approach behavior (either through innate mechanisms or learning).
–2) Increase the frequency and intensity of a behavior that leads to a reward (learning).
–3) Induce subjective feelings of pleasure (hedonia) and positive emotional states

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Dopamine and Reward

A

Many rewards lead to release of dopamine
(DA).
Dopamine is released during basic drives (i.e.,
hunger, sex).
DA is released in the rat right before and
during copulation, but not afterwards.
DA is released in the human when
presented with food stimulation in a food-
deprived state.
DA is released when participants are
playing video games for mone

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

The addicted brain

A
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Dopamine and Reward in the Brain

A
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Value representation in the human brain

A

Meta analysis of 206 fMRI studies examining neural correlates of subjective value
Two regions show a linear increase with subjective value:
Striatum and Orbitofrontal Cortex

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Role of the Striatum

A

The striatum plays a key role in learning
from reward value

The striatum plays a key role in coding
prediction errors allowing learning from
reward feedback to influence future choices

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Reward Prediction Error

A

Dopamine neurons are active when drop of
liquid is delivered outside any behavioral task.

Earliest predictor of reward signals dopamine
response instead of fully predicted reward

Dopamine neurons will be depressed at the
time of the predicted reward if it fails to
occur.

DA provides an error prediction signal to aid
in goal-directed behavior.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

prediction error hypothesis of dopamine

A

The idea: Dopamine encodes a reward prediction error

prediction error hypothesis of dopamine
p equals probability stimulus predicts delivery of reward.

No response to reward that is fully predicted (p=1),
response shifts to stimulus

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Do reward prediction errors drive learning in
humans?
Reward learning in humans

A
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Patients
w/ Parkinson’s disease

A

Loss of dopamine inputs to the striatum
Patients w/ Parkinson’s disease

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

FMRI

A

BOLD signal in striatum correlates parametrically, trial-by-trial with prediction
error (O’Doherty et al. 2003)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Learning reward value

A
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Prediction Error:

A

A general learning signal across
types of decisions
Example:
Social Reputation Learning
or
learning to trust in a behavioral economics games

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

The Trust Game

A
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

Hyperscanning:

A

two brains interacting &
learning to trust (or not)

23
Q

Striatum Response to Reward Feedback During the
Trust Game

A

Responses to monetary reward
in the striatum diminish and
shift to the presentation of
the trustworthy partner as
reputations are acquired
(and feedback is less
informative) during the
Trust game, i.e., prediction error coding

24
Q

Orbitofrontal Cortex

A
  • Clearly important in driving optimal
    decisions, but what is its role?
  • Evidence from Patients
  • Evidence from fMRI
25
Phineas Gage
Phineas Gage, (born July 1823, New Hampshire, U.S.—died May 1860, California), American railroad foreman known for having survived a traumatic brain injury caused by an iron rod that shot through his skull and obliterated the greater part of the left frontal lobe of his brain. Gage's case is considered to be one of the first examples of scientific evidence indicating that damage to the frontal lobes may alter personality, emotions and social interaction. Prior to this case, the frontal lobes were considered silent structures, without function and unrelated to human behavior.
26
Neural Systems of Decisions Orbitofrontal cortex: patient studies
* Base of frontal lobe resting on the orbit of the eyes Often broken down into 2 regions: – Ventromedial prefrontal cortex (in red) * This region was damaged on Phineas Gage – Lateral-orbital prefrontal cortex (in green) * Precise behavioral function is difficult to define – “irreverent, indulging at times in grossest profanity ... manifesting but little deference for his fellows, impatient of restraint or advice when it conflicts with his desires ... devising many plans of future operation, which are no sooner arranged than they are abandoned in turn for others.” – “Gage is no longer Gage” * Types of behaviors related to human orbitofrontal cortex may not be clearly analogous in other species * Patients with damage to this region seem to have difficulties with social and emotional “decision making”
27
Orbitofrontal Cortex Susceptible to Damage With Traumatic Brain Injury (TBI)
-fill an empty skull with gelatin -violently rotate the skull around (simulating a crash or some type of head trauma) -most of the “gelatin” brain remains smooth in appearance, except the orbitofrontal region. -caused by jagged ridges around the eye sockets, provide protective support for the eye orbits...
28
Neural Systems of Decisions: Orbitofrontal cortex patient studies- Patient E.V.R.
-orbitofrontal damage -could generate solutions to problems (e.g., two roommates trying to decide what show to watch) -could not distinguish which solutions were most likely to be effective -Damasio hypothesized that emotion was necessary to optimize decision making (“somatic marker hypothesis”) -showed some evidence of flat affect
29
James-Lange theory of emotion
emotion is your interpretation of your body’s physiological response
30
Damasio’s Somatic Marker Hypothesis (p. 400, Ward)
emotional information (physiological arousal) guides decision making -somatic marker = emotional reaction in your body (physiological arousal) -hypothesis: orbitofrontal region associates situations and somatic changes -current options evaluated based on likelihood to be rewarding
31
Initial Support for the Somatic Marker Hypothesis
-Healthy people, or those with damage to brain regions outside the orbitofrontal cortex learn to draw from the correct pile (pile B; drawing from pile A will result in net loss) -Are they learning by doing the math, tracking the probabilities/expected values? -Or does one pile “feel better” after a while, and so people “go with their gut”? -Healthy people, or those with damage to brain regions outside the orbitofrontal cortex learn to draw from the correct pile (pile B; drawing from pile A will result in net loss) -Patients with orbitofrontal damage do not figure out which pile to draw from. They also do not show increased GSR for risky draws. -they also show an increased galvanic skin response when anticipating a risky draw (from pile A) -Suggests that failure to generate somatic marker impairs ability to learn the task....(what more would be required to convince you?)
32
Evidence Against Somatic Marker Hypothesis
Patients with spinal cord injuries (no physiological feedback; i.e., no somatic markers) have normal emotional experience and can learn the Iowa Gambling Task just fine... Difficulty for frontal patients could be difficulty with updating value of the decks (changing rules). In fact, if the deck starts stacked so large losses are evident early, they learn to avoid “pile A”.
33
Neural Systems of Decisions: Orbitofrontal cortex: patient studies
Relative Value of the Choice in context: – One criterion for selection of an action is to weigh cues about appropriate response in a context. This function may be related to the orbitofrontal cortex (OFC) – Patients with damage to the OFC show a range of deficits in social decision making * Utilization behavior: patients will over rely on external, perceptual cues to choose a behavior even if it is not appropriate to the social context * Change in personality, irresponsibility, lack of concern for future consequences or social norms, diminished social awareness and empathy – Monkeys with OFC damage will show a decrease in social status – OFC in humans has been linked to antisocial personality disorder and aggression * Murderers have lower resting baseline metabolism in OFC than non-murderers – Most patients with OFC damage are not overly aggressive, but poor social decision making leads to problems in family, job, future planning
34
Orbitofrontal cortex Relative pleasantness
orbitofrontal cortex activity increases as pleasantness of temperature, taste, odors, physical attractiveness increases Also, chocolate, but OFC activity to chocolate decreases after eating chocolate (and participant is sated)
35
Orbitofrontal cortex Imaging studies summary
* BOLD response in the OFC correlates with perceived pleasantness * BOLD response in OFC correlates with subjective value, not prediction error * Combined with patients studies, is consistent with a role for OFC in encoding relative value of choice – Value considering the social/emotional context – Value in relation to previous value for the same choice (necessary to update value) – Value relative to internal states, preferences (perceived pleasantness), and long term goals – All key components of subjective value
36
Classical economist’s approach to decision-making
People are rational and should make the choice with the greatest expected outcome * However… people are not purely rational
37
Psychologist’s approach to decision-making
People don’t behave rationally because there are psychological factors at play– they use certain heuristics or have biases when processing decisions * Examples: People are risk averse because they treat gains and losses differently (i.e., “losses loom larger than gains”)
38
Neuroeconomics
Combines classical economic theory, psychological theory, and neuroscience
39
Value representation in the human brain
Meta analysis of 206 fMRI studies examining neural correlates of subjective value * Two regions show a linear increase with subjective value: * Striatum and Orbitofrontal Cortex
40
Getting oriented to the striatum
Sriatum is the largest structure of the basal ganglia * Dorsal striatum * Caudate nucleus * Putamen * Ventral striatum * Nucleus accumbens (NAcc) * Olfactory tubercle (OT)
41
Striatal “Reward Prediction Neurons”
* What do dopamine neurons in the striatum respond to? * Dopamine Response = Reward Occurred – Reward Predicted.
42
Striatal “Reward Prediction Neurons”
* What do Striatal neurons respond to? * Experiment: Monkeys is either given an unexpected reward or trained to associate a light with a reward a few seconds later * Result 1: When a monkey receives a reward without an expectation of reward, the neuron fires Result 2: A monkey is trained to associate a flash of light with a reward. When the predictor (light) is turned on, the dopamine neuron fires. The dopamine neuron does not respond to the reward. Dopamine neurons show little response when the reward is predicted and received * Result 3: When the predictor (light) is turned on, the dopamine neuron fires. When there is no reward, the dopamine neuron drops below baseline firing. Dopamine neurons show decreased activity when the reward is expected but not received (i.e., “negative prediction error”)
43
“negative prediction error”
Dopamine neurons show decreased activity when the reward is expected but not received
44
* Prediction error
Discrepancy between the expectation or prediction of reward and receiving the actual reward * In response to a positive predictive signals, they fire. * In response to a rewarding outcome, they : * Increase activity to “positive prediction error” when a reward is received but not predicted * Show little response when the reward is predicted and received * Show decreased activity when the reward is expected but not received (i.e., “negative prediction error”) * Prediction error signals in the striatum may be useful for learning about the rewarding stimuli in the world
45
A study of prediction error in humans
* Experiment setup: participants don’t know how rewarding a specific shape is. Each time, a shape is followed by nothing or a reward. So, they learn over time how rewarding something is
46
Which brain regions vary with reward prediction error
the striatum * Other evidence for the involvement of dopamine neurons (presumably in the striatum) in prediction error learning: * Parkinson’s patients are slower at learning from error feedback * Parkinson’s patients return to normal function with the L-Dopa * The striatum plays a key role in coding prediction error, allowing learning from feedback to influence future choices (across at least the domains of value and social reputation learning).
47
Orbitofrontal Cortex and Decision Making
Orbitofrontal cortex (OFC) is an important structure for emotional decision making Potential idea for the role of OFC in decision making: * Somatic Marker Hypothesis (Damasio)
48
Case Study: Patient E.V.R. (Elliot)
* Damage to OFC * Displayed dampened, dull emotional responses to situations * Can generate solutions to social problems and consider the consequences * … but cannot decide which solutions are most likely to be effective or which ones to choose * Damasio’s hypothesis: Maybe emotion is necessary to optimize decision making?
49
Damasio’s Somatic Marker Hypothesis
Key point: Our emotional reactions (i.e., physiological arousal) to a situation are “somatic markers” that can guide our decisions (e.g., “go with your gut”) * OFC helps ASSOCIATE situations and somatic (bodily) changes * OFC evaluates possible behavioral responses and their likelihood for reward, based on previous situations that elicited similar patterns of somatic change * Make decision based on experience that yielded best emotional response * Support from Iowa Gambling Task
50
* Support from Iowa Gambling Task
* Draw either pile A (riskier pile, both wins and losses are bigger) or pile B (safer pile, smaller wins and losses)à pile A results in net loss Healthy subjects eventually learn to draw from pile B * Healthy subjects also show increased skin conductance response (SCR) when drawing from pile A In contrast, OFC patients don’t show increased skin conductance response (SCR) when drawing from pile A, and don’t learn to avoid pile A OFC patients continue to choose pile A (riskier pile), and do not show SCR response from pile A * Suggests that failure to generate somatic marker impairs ability to learn from the task
51
* Evidence against somatic marker hypothesis
* Patients with spinal cord injuries don’t have physical feedback, and thus have no somatic markers Patients with spinal cord injuries don’t have physical feedback, and thus have no somatic markers * However… they learn the Iowa Gambling Task just fine à don’t need a physiological response to learn Maybe OFC patients have difficulty with cognitive control aspects of the task (e.g., changing rules) * Maybe they just have trouble overriding what they initially learned, which requires cognitive control * If the deck is stacked such that losses become evident early on, OFC patients can still learn to avoid pile A
52
Updating Reward Value/Weighing Value in Context * Key point: OFC may track the relative value of choice
* Value in relation to social/emotional context * Value in relation to previous value for same choice (updating value) * Value relative to internal states, preferences (perceived pleasantness) and long-term goals Evidence: Greater activity in OFC as pleasantness increases
53
OFC and Pleasantness
various temperatures of water on the hands OFC activity increases as pleasantness of temperature increases rich and delicious vs boiled vegetable water OFC activity increases as pleasantness of taste increases
54
OFC “Reward Neurons”
Thus, OFC may code value relative to the internal/external context, or code relative value Summary * Striatum = prediction error * Evidence from monkeys * Evidence from humans * OFC = relative value of a choice * Somatic Marker hypothesis * Updated view