Lecture 27 - Reinforcement Learning and Motor Sequences Flashcards
What is classical conditioning? What is the famous example?
a learned (reinforced) reflex/ response that is evoked by stimulus
Pavlov’s Dog
What is reinforcement?
increase behaviour
What is punishment?
decrease behaviour
Give an example of positive reinforcement.
Sweet treat
Give an example of negative reinforcement
take away homework
Give an example of positive punishment.
writing lines
Give an example of negative punishment
take away recess
What does the process of reinforcement learning involve?
Learning to link reward with specific actions (and their outcomes) so they become repeated
What is binary reward feedback?
Action is rewarded or not
What is a scalar quantity eward feedback?
relative to the utility of action/reward outcomes
What is the goal of reinforcement learning?
to maximize reward and minimize loss
What is cummulative reward?
it might be better to sacrifice immediate reward for long-term reward
What are some examples of cummulative rewards?
- chess
- investements
What happens to actions that are associated with reward?
they become strengthened/repeated
processes of reinforcement learning:
What is exploration?
the trial and error process of aquiring more information about the envrionemnt by searching possibilities
searchhing many action possibilites to determin which action tends to maximize reward
processes of reinforcement learning:
What is exploitation?
capitalize on known information to maximize reward
actions associated with past history of reward tend to be repeated to maximize future reward
What is the tradeoff between action exploration and exploitation?
shift emphasis from exploring to exploitation to maximize reward
What is a hockey example of the exploration, exploitation trade-off?
exploration - you find out goal tender is weak low
exploitation - shoot low to score goals (maximize reward)
What is the basal ganglia?
Collection of subcortical structures in the brain
Where is dopamine produced?
In the substantia nigra
What is dopamine?
a neurotransmitter that is part of the brains intrinsic reward system
basal ganglia:
What are the 2 parts of the striatum?
- caudate nucleus
- putamen
What is dopamine input into the striatum critical for?
learning from reward and strengtheing the representation of specific actions
Learning the piano example:
What happens to serial actions with learning?
can produce faster sequences with less errors. Key presses become smoother and linked together
What can practice do in terms of sequential actions?
Can link sequential actions into a single movement pattern
Chuncking of individual elements in a motor sequence
What happens in early practice?
shifting gears requires several dinstinct movements
Chuncking of individual elements in a motor sequence
What happens in middle practice?
independant actions are ‘chunked’ into a larger subunit of movement sequences
Chuncking of individual elements in a motor sequence
What happens in late practice?
actions become ‘chunked’ together into a cohesive movement sequence where successivce actions are ‘coarticulated’
What is chunking?
fusing a series of individual elements into a larger subunit of a movement sequence
What is co-articulation?
adjacent movement elements influence each other
What does co-articulation cause?
a blending of discrete movements within a sequence and is associated with an ncrease in speed and smoothness of movement sequences
What does sequence learning alter?
processing in several sensory and motor brain areas
Early stages of learning:
What is the role of the dorsolateral prefrontal cortext?
- working memory, attention, action selection
- conscious planning and processing of movements
- likely involved in intial stages of chunking motor sequences
Early stages of learning:
What is the role of the inferior parietal cortex?
processing visual and somatosensory information
Early stages of learning:
What is the role of rostral premotor areas?
- planning motor actions
- processing visual and somatosensory feedback
Early stages of learning:
What is the role of basal ganglia?
processing reward during action
Later stages of learning:
What is the role of supplementary motor area?
- selecting and planning previously learned motor responses
- likely involved in later stages of motor chunking and coarticulation
Later stages of learning:
What is the role of the dorsal premotor area?
- processing visual and somatosensory information
- involved in selecting learned actions
Later stages of learning:
What is the primary motor cortext involved in
producing motor actions
Early stages of learning:
What is the cerebellum involved in?
detecting and correcting errors
How does brain activity evolve during motor sequence learning?
Early stages require high cognitive demand and conscious prcoessing.
Later stages require low cognitive demand and automatic processing.
