Learning

Question 1

where is rate related processing evident

Answer 1

S1
PPC
globus pallidus internal segment

Question 2

what amount of all mvmt related neurons correlate with the direction of limb mvmt but not with the velocity of mvmt

Answer 2

30-50%

Question 3

where is mvmt related activity found

Answer 3

motor related portions of the basal ganglia (STN, putamen) and thalamus

Question 4

where is rate related activity restricted to

Answer 4

globus pallidus internal segment
- speed and vigor of mvmt

Question 5

what is the process of reinforcement learning

Answer 5

involves learning to link reward with specific actions and their outcomes so they become repeated

Question 6

what is the difference between error feedback and reward feedback

Answer 6

reward feedback = binary (action is rewarded or not)
error feedback = isn’t binary

Question 7

what is the goal of reinforcement learning

Answer 7

maximise reward and minimise loss
- actions that are associated with reward become strengthened/repeated to maximise reward

Question 8

what is cumulative reward

Answer 8

idea that it might be better to sacrifice immediate reward for long term reward
- ex: chess, investments

Question 9

what is the exploration process in reinforcement learning

Answer 9

trial and process of acquiring more info about the environment by searching possibilities
- searching to determine iwhich actiosn tend to maximise reward

Question 10

what is the exploitation process in reinforcement learning

Answer 10

capitalise on known info to maximise reward
- actions associated with past history of reward tend to be repeated to maximise future reward

Question 11

what is the trade off between action exploration and exploitation

Answer 11

shift emphasis from exploring to exploiting to maximise reward
- exploration = find out goaltender is weak low
- exploitation = shoot low to score more goals

Question 12

what is the function of the basal ganglia in reinforcement learning

Answer 12

dopamine = critical for brains intrinsic reward system
dopamine input to the striatum is critical for learning from reward and strengthening the representation of specific actions

Question 13

when are the dopaminergic neurons in the substantia nigra pars compacta most active

Answer 13

time locked to the presentation of reward
- high firing rate right after reward onset

Question 14

what is the reward related activity in dopaminergic neurons

Answer 14

activity of dopaminergic neruons scales with the amount of reward
- increase in response of dopaminergic neurons with reward

Question 15

what is the temporal discounting of reward related activity in dopaminergic neurons

Answer 15

as time to reward increases, amount of dopamine released with reward presentation decreases

Question 16

where does temporal discounting occur in the phases of response

Answer 16

valuation phase (where the value of reward is evaluated)

Question 17

what is the relationship between rate of learning and probability of reward

Answer 17

rate of learning increases when the probability of reward is higher
- faster learning but levels out to the same max rate

Question 18

when does a positive reward prediction error occur

Answer 18

before learning
- unconditioned (no predictive stimulus)

Question 19

when does no reward prediction error occur

Answer 19

when reward is predicted AND provided
- after learning (conditioned)

Question 20

when does a negative reward prediction error occur

Answer 20

reward is predicted but omitted
- after learning (conditioned)

Question 21

what occurs to synapses based on presence and absense of reward

Answer 21

presence = strengthens synapses that are invovled in generating the rewarded behaviour
absence = weakens synapses that were involved in generated with rewarded behaviour