PSYC 361 MT2: DECISION MAKING & GAMBLING (3)

Question 1

What is Decision Making?

Answer 1

Process of selecting particular option among set of alternatives with different expected outcomes, typically carried out to maximize certain desirable outcomes
- extremely broad range of behaviours: taxes to political behaviour
- goal-ducted at core, prediction-experience involved

Reason vs Passion
- Normative Approach: optimal decision based on utility maximization; use economic principles (ie. expected utility theory & game theory) to predict logical decision

• Empirical Studies: actual animal behaviours more often than not violate predictions— suggests decision making influenced by other factors
  —> Prospect Theory (Kahneman, Tversky): concerning decision making under risk:
  ———> Certainty Effect: underweight merely probably outcomes compared to certain ones
  ———> Isolation Effect: inconsistent preferences for same choice presented in different forms
  ———> Loss Aversion: prioritize minimizing losses over maximizing gains

Question 2

Neuroeconomics

Answer 2

Developed from behavioural economics, combined with neuroscience & other disciplines of psychology

Interdisciplinary field to explain human decision making; ability to process multiple alternatives & follow through on plan of action

To investigate how brain processing info for optimal decision making

Question 3

Decision Making has Different Stages

Answer 3

1) Perceptual Decision Making: analyze sensory info to recognize objects

2) Cognitive Decision Making: compare alternatives & make plan of action

3) Motor Decision Making: carry out action with proper motor program

Question 4

PFC in Decision Making

Answer 4

Phineas Cage: PFC critical for decision making
- brain damage led to dramatic changes in personality: capable, efficient, smart to irreverent, profane, socially inappropriate, poor decision-making— unable to work; still had normal motor & language skills

Question 5

Brain Areas Involved in Decision Making

Answer 5

• PFC, Nucleus Accumbens, VTA

DAergic Pathway connects them all
- Decision making altered in patients with neuropsychiatric/neurodegenerative disorders

Impulsivity: tendency to respond prematurely in risky fashion (SCZ, ADHD, OCD)
- Waiting & risky choice impulsivities: DA & 5-HT in NAcc + PFC-NAcc projections
- Stopping Impulsivity: dorsal striatum, orbitofrontal cortex, dorsal prelimbic cortex
- NE reputable blocker ameliorates waiting impulsivity in NAcc shell & stops impulsivity in PFC

Reward Encoding
- DA system: substantia nigra, VTA, basal ganglia (striatum)
- Parkinson’s patients sometimes develop iatrogenic gambling problems with L-dopa

Question 6

Brain Areas Involved in Decision Making- CORTICAL

Answer 6

Cortical Contribution: perception-action cycle
- 3 Levels:
1) Hypothalamus/ANS (physiological)
2) Limbic (emotional & value)
3) PFC (executive)
- continuous processing of internal & external info to guide decision making

Question 7

Social Decision Making

Answer 7

Group-living animals need to make decisions with other members in consideration— reflects social values of behaviours

Influenced by environment, context:
- Theory of Mind: knowing others’ intentions &/ ideas
- Cultural/Societal Norms: knowledge regarding acceptable & unacceptable behaviours
- Social Relationship: artificially increasing oxytocin level biases trust behaviour
- Social Status: dominant, subordinate

Question 8

Procrastination

Answer 8

Represents decision-making problem: choosing between getting task done & relaxing/having fun— decision making results in delay in task completion:
- Arousal Seekers: some report procrastination motivated by need for deadline-triggered arousal (to enhance performance) (Simpson, Pychyl)

• Emotional regulation difficulties contribute to procrastination (Bytamar)
• Attention: people tend to pay more attention to interesting stimuli & avoid less interesting/repulsive stimuli
  —> ADHD population has stronger tendency to procrastinate
• Learning Effect: reinforcement by satisfactory performance from previous procrastination (Magnus)
  —> conditioned reinforcement may play role

Question 9

Gambling Requires Decision Making

Answer 9

Gambling: wagering something of value (stakes) on random event with intent of winning something of value; instances of strategy discounted
- Consideration: wagered amount (cost)
- Risk: probability of win/lose
- Prize: expected outcomes (benefit)

Question 10

Somatic Marker Hypothesis (1)

Answer 10

Motivated behaviour influenced by emotions; emotional responses involuntary & often hard to fake— useful predictive indicators of behaviour

Somatic Marker Hypothesis (Damasio)
- We use emotion-based signals to guide decision-making
- When considering particular +/- outcome, emotional state is generated which helps to incentivize/suppress certain options in decision making process
- Hypothesized to indicate value, boost attention/WM & reduce problem space by marking options with emotional signal
- Failure to generate/appreciate meaning can lead to deficits in affective learning & goal-directed behaviour

Question 11

Somatic Marker Hypothesis (2)

Answer 11

Arise from periphery/central representation of periphery— critical neural substrates postulated to be in PFC:
- Receives projections from sensory & somatosensory areas
- Receives projections from bioregulatory systems: hypothalamus & neuromodulator inputs (DA, 5-HT, ME, Ach)
- Heavy projections to & from amygdala
- Involved in learning about contingencies
- Affects motor system through activating premotor cortex & basal ganglia
- Influences ANS via thalamus & hypothalamus

Question 12

Iowa Gambling Task (IGT) (Bechara)

Answer 12

To represent real life decision making situations involving uncertainty, reward & punishment
- participants to choose card for 100 trials, must forego ST benefit for LT profit
- value/goal: win as much money
- reward/punishment: not obvious
- “intuitive” decision making relied on
- learning occurs over trials
- somatic markers?

NORMAL IGT PERFORMANCE:
- healthy controls learned to avoid disadvantageous decks & choose advantageous more; skin conductance responses (SCRs), somatic marker associated with successful learning

BRAIN LESIONS DISRUPT PERFORMANCE:
- amygdala & VMPFC implicated in emotion processing; showed impaired IGT performance

Question 13

IGT: Failing to Generate Somatic Markers Impairs Performance

Answer 13

Anticipatory SCRs: measured in time period before card is turned over (~5s)
Reward & Punishment SCRs: measured in 5s after card is turned over

VMPFC LESION ALSO IMPAIRS LEARNING EFFECT ON IGT:
- healthy controls learn to optimize card choices over repeated testing
- VMPFC patients still fail to learn after 6. Months if not worse

Question 14

IGT Performance Supports the Somatic Marker Hypothesis

Answer 14

Poor decision making associated with impaired ability to generate somatic markers

Performance of lesioned patients aligns with predictions form hypothesis

Question 15

Limitations of the Somatic Marker Hypothesis

Answer 15

1) Healthy volunteers show similar SCRs regardless of IGT performance:
- Ps bad at IGT still developed aSCRs, suggests aSCRs not sufficient for optimal decision making
—> P’s anecdotally “risk seekers”— at risk population?
- Individual differences in IGT performance in healthy volunteers:
—> only good performers showed elevated HR & SCR before making risky choice
—> some Ps learned task did not develop aSCRs; suggests not necessary for optimal decision making

2) Hard to determine causal relationship:
- SCRs may be corresponding to greater variance in outcome (bigger differences between gains & losses in disadvantageous decks)
- reward-elicited SCRs in monkeys not modulated by reward size & peaked after target stimulus selected
—> SCRs may be associated with reward anticipation & occur after excision has been made than drive decision itself

3) Spinal cord injury patients show similar IGT performance
- “body loop” disconnected, but “as-if loop” may still function to mediate effect?

Question 16

Learning in Decision Making

Answer 16

Law of Effect: specific responses strengthened/weakened by outcomes
- response choice essentially problem of decision making

New environmental info can be used to improve outcomes of choices

Learning increases behavioural flexibility for optimization of decision making
- V1 activity changes correlated with performance improvement
- PFC activity corresponds to “decision” variable predicted by learning

Learning also modulates decision making in long-lasting fashion

Question 17

Reinforcement Learning Theory in COMP NRSC

Answer 17

Reinforcement Learning Theory (NOT reinforcement in operant conditioning)
- in Pavlovian conditioning, CS predicts US (outcome); in SR habit learning (instrumental model-free reinforcement learning) emitted action predicts outcome
- Reward Prediction Error: difference between expected outcome & experienced outcome
- new info updates a “mental model” (model-based reinforcement learning) & a decision is made by simulating outcomes excited form this model

Neural substrates for model-free & model-based reinforcement learning:
- Model-Free (habit): DA system, striatum, motor area, PFC
- Model-Based (goal-directed): dorsomedial striatum, prelimbic cortex, hippocampus, PFC (orbitofrontal & medial prefrontal), posterior parietal cortex

Question 18

Reversal Learning in Gambling Tasks

Answer 18

Reversal Learning: modulation of behaviour to obtain reward when reward contingencies change
- initially animals learn to favour CS+ associated with reward & avoid CS- not associated with reward; when contingencies reversed, CS+ & CS- switch associative relationships with reward

(Follows, Farrah): patterns of cards associated with $50 gain/loss; 2 cards are dealt each time. Ps asked to make choice & provided feedback about outcome; after 8 consecutive winning choices, contingencies reversed.

Participants are expected to make few errors when contingencies area reversed— indeed case for healthy controls & patients with dorsolateral PFC (DLF) lesion; however VMPFC lesion leads to more difficulties

Question 19

Reversal Learning in IGT

Answer 19

Decks A&B associated with initial large reward & considered CS+ but over multiple trials, Ps learned A&B are associated with much larger punishment (disadvantageous decks result in net loss), reversing contingencies & CS+ now becomes CS-

Patients with VMPFC lesion showed impaired IGT performance as in previous case

Question 20

Reinforcement Learning in IGT

Answer 20

(Worthy): computationally simulated IGT data with different learning-based strategies
- Win-stay/lose-shift (WSLS) strategy sensitive only to outcome of previous choice; strategy provides best fit to IGT data from about half of human data
- Prospect Valence Learning (PVL) model utilizes decay reinforcement learning rule (cumulative rewards) provides best fit to other half of human data

(Wyckmans): analyzed real participant’s’ peformance (of different gambling task) with reinforcement learning model, simulating decision making results predicted by model-free & model-based reinforcement learning
- found model-based decision making reduced in problem gamblers compared to healthy volunteers

Question 21

Summary of Decision Making & Gambling

Answer 21

Decision making has several stages & involves multiple process, hence, normative predictions of decision making are often violated by real-world behaviour

The hypothalamic-limbic-PFC pathway is heavily involved in decision making

Somatic marker hypothesis argues that we rely on the emotional info to bias our decision

IGT performance both support & challenge the somatic marker hypothesis

Learning plays key role in decision making

Computational NRSC enriches our toolbox to further our understanding of behaviour