lecture 9- dopamine, reward & reinforcement

Question 1

what is dopamine?

Answer 1

a type of neurotransmitter

Question 2

what is the importance of dopamine?

Answer 2

important for learning, cognition and behavior.

important for coding prediction errors which drives our learning as we only learn things when we are surprised.

Question 3

what is a negative reinforcer?

Answer 3

decrease the frequency/probability of a certain action- avoiding doing the action, not a punishment just avoidance.

Question 4

what is an example of negative reinforcement?

Answer 4

you tidy up the house to avoid getting shouted at by housemate.

Question 5

what is a prediction error?

Answer 5

errors in the prediction of rewards signal the inappropriate/appropriate nature of the actions performed to obtain them.

Question 6

give the 2 examples of Reward Prediction Errors (RPEs)

Answer 6

1. unexpected rewards

2. failure to obtain a predicted reward

Question 7

why are RPEs important?

Answer 7

outcomes are unexpected and therefore are important in driving our learning.

Question 8

explain the Rescorla-Wagner model.

Answer 8

basically explains the various forms of classical conditioning reliably.

Question 9

why would it be good to find neurons that respond to reward stimuli?

Answer 9

if we find neurons that encode the unpredictability of the reward it would be very beneficial for learning.

Question 10

dopaminergic neurons have what receptors? and what is the release of dopamine associated with?

Answer 10

dopamine receptors, and it is associated with pleasure.

Question 11

what pathways in the brain are associated with dopamine?

Answer 11

1. Mesostriatial- basal ganglia which influences motor control.
2. mesolimbocortical pathway -including hippocampus, which influences memory and reward processing.

(most are middle/front of brain effected by dopamine).

Question 12

where contains a large proportion of dopamine neurons?

Answer 12

midbrain- Ventral tegmental area (VTA) .

Question 13

where do dopamine neurons project to?

Answer 13

Striatum and anterior cingulate cortex.

Question 14

what do dopamine neurons respond to?

Answer 14

presentation of rewards.

Question 15

an example of a prediction error is with monkeys and apple juice. explain the levels of dopamine throughout this experiment.

Answer 15

before the task- dopamine neurons respond to unpredicted rewards.
during learning- reward became increasingly predictable.
when the reward becomes more predictable less dopamine is fired.
if the monkey doesnt get juice- no dopamine is fired- predictable rewards elicit no change.

Question 16

what happens if rewards are delivered at the wrong time ?

Answer 16

there is a decreased activity at the time that rewards are expected, followed by increased activity at the time of delivery. (decreased dopamine when reward not given when they think they are getting a reward).

Question 17

what do Schultz et al (1997) suggest about reward prediction?
(in monkeys remember)

Answer 17

during learning the dopamine response switched from the reward (US) to the reward predictor (CS). rewarding experienced at time of Que not actually when given reward.
neurons also involved in reward prediction - Que itself could be like a reward itself.

Question 18

VTA in humans - D’ Ardenne et al (2008) experiment explain…

Answer 18

P’s played a card guessing game - guess whether the number on the right would be bigger/less than on the left- they won $1 if guessed correctly and lost $1 if guessed incorrectly.
results= activity in the VTA increased when they UNEXPECTEDLY got the $1.. and was greater as the probability of the reward increased.
therefore some evidence for prediction errors in humans however still debated (Duzel et al, 2009) as there was NO response for negative prediction errors.

Question 19

what is the equation for the dopamine response??

Answer 19

dopamine response= actual reward- predicted reward.

following Rescorla- Wagners learning principle.

Question 20

what does Matsumoto et al (2013) suggest about VTA dopamine neurons signalling cognitive motivation?

Answer 20

experiment in monkeys:performing a delayed matching-to-sample task that required working memory and visual search.
results= dopamine neurons responded to task events associated with cognitive operations.
The response of dopamine neurons in the ventral tegmental area is modulated by task difficulty!!

Question 21

what did Walton et al (2007) show?

Answer 21

dopamine neurons coding how much effort is needed to put in for output.
Coding for = Reward - Amount of effort

Question 22

where are ‘reward expectation’ neurons found in monkeys and rats brains?

Answer 22

the Striatum.

(part of the basal ganglia) also known as the ICE CREAM CONE.- the nucleus accumbens

Question 23

what two parts in the Stratium composed of?

Answer 23

1. The Caudate

2. The Putamen

Question 24

what did Pessiglione et al (2006) show?

Answer 24

evidence for prediction errors in humans by looking at the response in the stratium from fMRI.
L-DOPA (enhances dopamine) and haloperidol (reduces dopamine) was administered during instrumental learning.
L-DOPA= ehanced the RPE signal and choose the most rewarding action relative to subjects treated with haloperidol.

Question 25

where else do dopamine neurons target in the brain??

Answer 25

Cingulate cortex- anterior and mid portions
respond to rewarding stimuli also.
Anterior Cingulate Cortex (ACC).

Question 26

what did Kennerley et al (2011) find in monkeys ACC when making choices between rewards?

Answer 26

they found neurons that responded to +ve, and -ve prediction errors.
in addition to neurons responding to both at the same time.

Question 27

Ribas-Fernandes (2011) examined RPEs in HUMAN ACC’s what did he find?

Answer 27

after examining human fMRI scans after performing a task where there is a sub goal leading to an overall outcome, they found activity in the ACC that signalled prediction errors for the individual actions leading to a goal.

Question 28

what happened to the rats that were hooked up to a self administration of dopamine?

Answer 28

if the rat pressed the lever more dopamine was released, rats ended up passing out because they repeatedly released the dopamine for pleasure.

Question 29

how does cocaine work on dopamine?

Answer 29

directly on dopamine neurons by blocking dopamine re-uptake. increasing the concentration of dopamine at the synapse - sense of euphoria.
(showing dopamine is rewarding).
cocaine effects transmission- dopamine cant go back to be recycled in the pre synaptic membrane.

Question 30

cocaine and temporal coding: what happens when rats are assked to wait for a reward when on drugs?

Answer 30

cocaine rats cannot wait long enough, they are too motivated for getting the reward + are expecting the reward so much they cannot wait to press the lever.

Question 31

summary - the Mesolimbic Dopamine system what are the 4 main points to this?

Answer 31

1. learns to predict rewards following PE’s.
2. signals the anticipation of an upcoming reward.
3. motivates the performance of action for getting rewarding outcomes.
4. disturbed by drugs- which change motivations.

Question 32

what does Behrens et al (2008) suggest about PE’s?

Answer 32

learning about others may be underpinned by similar reinforcement learning processes.
we make predictions what other peoples actions may be in a similar way to how we make predictions about ourselves.
therefore we can correct predictions about others when the prediction was in error.

Question 33

discuss Behrens et al (2008) experiment on PE’s beyond reward learning looking at social cognition…

Answer 33

looked at humans fMRI, whilst they performed a decision making task which required them to see which is the best out of two options and were rewarded.

• on each trial they recieved advice outside of the scanner.
• subjects used the probability of a reward being received and the probability of the advice not being true (social learning).
• results found: activity in the nucleus accumbens/ ventral stratium signalled the RPEs for the subjects own learning about rewards.

Question 34

Kennerley, Behrens & Wallis (2011) found a double dissociation between which two types of neurons?

Answer 34

Orbitofrontal neurons and anterior cingulate cortex neurons.