Decision- Making (part 2) Flashcards
Reinforcement
Learning
We perform actions in the world and experience
the resulting outcomes as good (reward) or bad
(lack of reward or punishment)
Rewards and the Brain
Old idea about dopamine and rewards:
Midbrain dopamine system signals
pleasure/reward.
BUT midbrain dopamine system does not
simply signal reward:
What does dopamine signal? And how
was this shown?
Learning Driven by Rewards
Single unit recordings from
monkey’s midbrain dopamine
neurons in ventral tegmental
area
Conclusions Learning Driven by Rewards
Conclusions:
* Activity of midbrain dopamine neurons is
related to reward
* But dopamine neurons do more than
simply report occurrence of reward
* They code deviations from predictions
about time and magnitude of reward
Reward Prediction Error (RPE) =
Actual Reward – Expected Reward
RPE > 0, Better than expected
RPE = 0, As expected
RPE < 0, Worse than expected
Dopamine Pathways in Human Brain
Midbrain dopamine
neurons project to
basal ganglia,
prefrontal cortex, and
many other areas
Reinforcement
Learning &
Addiction
- Opioids physiologically
trigger release of
dopamine - This is misinterpreted as
a reward prediction error
signal - Thus, opioids “hijack”
the reinforcement
learning mechanism
Iowa Gambling Task
a psychological test used to assess decision-making under uncertainty and risk. It was designed by Antonio Damasio and colleagues to study how people make choices that involve both rewards and penalties over time
VMPFC & IGT, Iowa Gambling Task (IGT)
The Iowa Gambling Task (IGT) assesses decision-making by choosing between decks with different reward/penalty structures.
VMPFC & IGT:
The VMPFC is key for emotional processing and decision-making.
VMPFC damage leads to poor decision-making, as patients fail to avoid disadvantageous decks, focusing on immediate rewards over long-term consequences.
Emotion & decision making
Iowa Gambling Task (Bechara et al., 1994)
* Conceptual knowledge alone did not lead to good decision making
* Predictions of expected emotions (somatic markers) were necessary for
good decision making
* Expected emotions preceded conceptual understanding
* Patients with damage to VMPFC could not generate expected emotions
* Patients overemphasized immediate reward over long-term outcomes
(temporal discounting)
prefrontal
cortex
Functions include:
* Maintenance and updating of goals
* Inhibition of prepotent actions
* Shifting between rules, sets, and tasks
* Monitoring and adjusting performance
* Integrating multiple sources of value
All central to decision making and the
selection of actions
Rostral abstraction
Rostral PFC
* Complex
* Abstract
* Long timeframe
Cauadal abstraction
PFC
* Simple
* Concrete
* Short timeframe
Rostral/Caudal:
Abstraction
Example (alley-oop!):
* Rostral: What sport am I
playing?
* Basketball
* What position am I playing?
* Forward
* What play are we running?
* Alley-oop
* Caudal: What do I need to
do next?
* Jump!
Ventral PFC
What
* Why
* Meaning oriented
On offense
* Need to put ball through hoop
to score
Dorsal PFC
Where
* How
* Action oriented
Approaching basket
* Catch ball at apex
* Slam it through the hoop!
Medial PFC
Hot/affective/motivation
* Value-based
* Internal/body-oriented
Need to score to win!
* Don’t want to make a mistake
* Completing an alley-oop feels
good
Lateral PFC
Cold/cognitive/rules
* Feature-based
* External/environment-
oriented
Positions of players allows it
* High-percentage shot
* Quick play needed due to shot
clock
Organizational principles of prefrontal cortex
the part of the brain involved in higher-level functions such as planning, decision-making, problem-solving, and controlling emotions and behavior. It is organized in a way that different regions handle specific tasks:
