Week 6 (Chapter 47, 48) Flashcards
What is thought to be at the centre of reinforcement learning in the mammalian brain?
The midbrain dopaminergic system
What are the uses for a theoretical framework for reinforcement learning?
- aid in the interpretation of neurophysiology data
- guide the design of future studies
The Rescorla-Wagner model formalized the idea that _______ is needed to drive learning
An error between the actual and predicted outcome
What is the learning theory originally formulated for computer learning?
Temporal difference (TD) learning
Reward expectation reduces dopamine reward responses in a purely _____ fashion
Subtractive
A classic adaptation of TD learning to a biological circuit model utilizes a _____
Complete serial compound (CSC) feature representation
Dopamine neurons show a negative prediction error ______
At the time of an expected reward
Dopamine responses to reward are suppressed only at _____
The time of the expected reward
What happens when the timing of a reward is shifted?
A larger dopamine response
Cues followed by late rewards result in _____ dopamine responses than cues followed by early rewards
smaller
What TD model was proposed to explain the shortcomings of the CSC TD model?
The microstimulus model
How does the microstimulus TD model differ from the CDC TD model?
The microstimulus model is able to account for the longer dip of dopamine responses upon reward omission
In 1998, Hollerman & Shultz discovered that when monkeys were given a reward stimulus earlier than expected, ______
There was a large dopamine response upon the reward, but no negative response at the time of the usual reward
What is one possible modification to the CSC TD model after Hollerman & Shultz’s discovery?
An animal is in two states - ISI when expecting a reward and ITI when not. When one is activated, the other is deactivated
Semi-Markov dynamics imply that time spent in a state is _____ and is defined by ______
- probabilistic
- a probability distribution called a ‘dwell time distribution’
