Riley Dopamine 4

Question 1

What do we still not know about reward?

Answer 1

The features of rewarding stimuli that make VTA neurons fire

Question 2

What did Wise et al. (1978) suggest?

Answer 2

That midbrain DA release mediates the hedonic aspects of reward

Question 3

Who is Wolfram Schultz and what did he propose?

Answer 3

One of the top neuroscientists, over 11,000 citations for his 2 seminal papers
Schultz (1997,1998) suggested that midbrain DA release represents a reward prediction error

Question 4

What is the first basic function of rewarding stimuli?

Answer 4

Rewards elicit approach and consummatory behaviour and serve as goals of voluntary behaviour
They interrupt ongoing behaviour
Change the priorities of behavioural actions in order to gain maximum utility of rewards

Question 5

What is the second basic function of rewarding stimuli?

Answer 5

Rewards have positive reinforcing effects and thus increase the frequency and intensity of behaviour leading to such objects
Learning happens when rewards occur unpredictably and slows as rewards become more and more predicted
Reward-driven learning depends on discrepancy or ‘error’ between the prediction of reward and its actual occurrence

Question 6

What is the third basic function of rewarding stimuli?

Answer 6

Rewards induce subjective feelings of pleasure (hedonia) and positive emotional states

Question 7

What happens when an action produces an unpredicted outcome?

Answer 7

If it produces an unpredicted positive outcome, it is likely to be repeated.
If it produces an unpredicted negative outcome, it is unlikely to be repeated

Question 8

What happens when a reward is predicted?

Answer 8

When a reward is entirely predicted by sensory cues, no further learning takes place

Question 9

Whats an example of RPE in real life?

Answer 9

Trying out a new restaurant, if unpredicted positive outcome - will go there again
If unpredicted negative outcome - won’t come back again
But if these things are predicted then no learning will happen

Question 10

How did Schultz investigate if the RPE is reflected in midbrain DA neuron activity?

Answer 10

Recorded extracellular activity from midbrain DA neurons (VTA and substantia nigra) while monkeys performed a behavioural task
The naive monkey was required to touch a lever following presentation of a signal light, giving a juice reward in return
After several days of training, animals learnt to reach for the lever as soon as the light came on

Question 11

How is data presented from neurons firing?

Answer 11

Raster plots
Raw output representing the firing of neurons in time
The horizontal rows correspond to a single trial
Each dot or bar corresponds to a spike (AP) observed during the trial
The trials are aligned to stimulus onset

Question 12

How do midbrain DA neurons respond to reward before learning?

Answer 12

A drop of liquid occurs although no reward is predicted at this time
This constitutes a positive RPE
Midbrain dopaminergic neurons fire and are activated by the unpredicted occurrence of the reward

Question 13

How do midbrain DA neurons respond to reward after learning?

Answer 13

A conditioned stimulus predicts reward and the reward occurs according to prediction so therefore is no RPE
Midbrain dopaminergic neurons are now activated at the presentation of the CS and the response is not seen after the reward due to no RPE present

Question 14

What happens if the CS is presented but the reward fails to occur?

Answer 14

This constitutes a negative RPE
Shows an increase in DA neuron activity after the CS presentation due to a conditioned expectation
But a dip is seen in DA neuron activation (which is tonic) due to the absence of the predicted reward

Question 15

What does the RPE act as?

Answer 15

A teaching signal that is used to correct inaccurate predictions

Question 16

How did Schultz operationalise RPE?

Answer 16

dopamine response = reward occurred - reward predicted

Question 17

How can conditioning be impaired in regards to blocking?

Answer 17

Conditioning is impaired if a new CS is presented together with a second CS that has already been associated with the response
CS1 fully predicts the arrival of a reward
So CS2 provides no change in RPE and thus no learning takes place

Question 18

What happens during blocking in the midbrain DA response?

Answer 18

The blocked stimulus doesn’t become associated with the reward so doesn’t yield a DA response like CS1 does

Question 19

What happens if hypothetically CS1 = no reward, and the addition of CS2 lead to a reward?

Answer 19

Then the addition of CS2 would produce a positive RPE and associated DA response
The unblocked stimulus (CS2) will then become associated with reward and produce a DA response on its own without CS1

Question 20

Outline blocked stimulus DA response

Answer 20

If a blocked stimulus produces no reward then there is no change in DA firing but if it does produce a reward then there is an increase in DA firing

Question 21

Outline unblocked stimulus DA response

Answer 21

If an unblocked stimulus doesn’t produce a reward there will be a dip in DA firing, and if there is a reward there is no change

Question 22

Outline the stepwise transfer of the DA response

Answer 22

A transfer of the DA response to earliest predicting stimuli
This is how chains of behaviours can be rewarding, by learning the patterns of behaviours which lead to reward.
This transfer allows for more complex learning

Question 23

Outline RPE and risk taking

Answer 23

Many naturalistic rewards contain elements of risk, which is reflected in the DA neurons

Question 24

How is the RPE/risk taking relationship studied?

Answer 24

Risk is measured in a lab using binary gambles
A monkey is cued to make a reward-based decision
1) 50:50 chance of a small/large amount of juice
2) Guaranteed a small amount of juice
By increasing the size of reward we increase the risk

Question 25

What did the monkeys show?

Answer 25

Non-linear preferences for rewards
When risks are low they gamble for large rewards
But when risks are high they take the guaranteed small reward

Question 26

What does the monkeys midbrain DA response show?

Answer 26

The same non-linearity
Successful gamble produces positive RPE
A higher risk gamble is associated with high midbrain DA activity associated with the cue and reduced midbrain DA activity at the time of the reward
This could be a moderating evolutionary effect, linked to risk management at the level of the DA neurons?
The highest increase in midbrain DA activity was at the time of reward in medium risk conditions - not high

Question 27

What could be theorised about gamblers?

Answer 27

That this risk management/moderating system breaks down in gamblers

Question 28

What is one concern about the RPE hypothesis?

Answer 28

That it is too quick (150ms) to be a meaningful signal about stimuli

Question 29

What has recent evidence shown about the midbrain DA response?

Answer 29

That it is comprised of 2 phases:

1. An unselective transient phase that cues for any potentially rewarding salient object
2. A second phase that codes for the reward value of a stimulus

Different populations of neurons?

Question 30

So does DA release at the NAc represent the RPE?

Answer 30

Yes it does
Fast-scan cyclic voltammetry (like microdialysis but quicker) measured reward-evoked DA release in the NAc (Hart et al., 2014)
DA concentrations in the NAc display a bi-directional RPE signal
- symmetrical encoding of positive and negative RPEs
- DA release increases with positive error
- DA release decreases with negative error

The more unexpected the larger the increase/decrease

Question 31

What are the first implications of RPE for processing in the NAc?

Answer 31

The NAc integrates input from cortical and subcortical sites, often synapsing onto the same neuron that VTA targets
Coincidental cortical NAc synaptic activity with the VTA appears to strengthen the conditioned responses - probably mediated by plasticity
Dopamine release is required for the corticostriatal plasticity
Positive RPE = higher conc. of DA released = increased likelihood of potentiation

Question 32

What are the second implications for action selection in the basal ganglia?

Answer 32

The NAc projects to the basal ganglia to initiate motor programmes in the pursuit of reward
Motor programmes are initiated based on the strongest ‘bid’ for activity at any given time
When we have a positive RPE a stronger bid is put into the BG - behaviour motivated by that reward is more likely to be initiated

Question 33

How can optogenetics be used in future research?

Answer 33

Insert a genetically encoded opsin into a rat, these are coded so that only specific neuron types accept it, providing specific control
Then shine a light into the brain at a given wavelength and the neurons express the opsin as they would to a natural event
Can use this light to get controlled and consistent responses from the midbrain neurons