Unit 4 Flashcards
chaining builds complex R sequences by linking together S –> R –> O
- initially train animals perform first behavior
- reward for performing 2 sequential behaviors
- allows series of behaviors
chaining
____ through successive approx builds a complex single R incrementally
- initial contingency is introduced for simple behavior, R
- as rate of R increases, contingency is moved to a more complex version of R
- gradually builds a complex R animal would never spontaneously produce
shaping
closer or longer in time the behavior and consequence –> better the learning
closer
every X Rs produces 1 O
Fixed Ratio
lower or higher ratio –> longer pause after each reward
ratio
time out from responding after each reward
post reinforcement pause (flat line)
every X Rs produces 1 O, but X changes with each delivery of reinforcer
- identify by avg number of Rs per O
variable ratio
constant and high rate of responding
- gambling
- video games
- sports
variable ratio
After Y secs, 1 R produces 1 O
fixed interval
After Y seconds, 1 R produces 1 O, but Y changes after each O
variable interval
more responses = more reinforcers
gotta play to win
VR
more responses doesn’t necessarily mean more reinforcers
only need to check in
VI
strong start, short delay btw R & O, FR1 schedule, other Rs available, contingency always in effect
effective punishment
links stimulus with response (S–> R)
dorsal striatum of basal ganglia
(caudate nucleus + putamen)
linking responses with outcomes (R–>O)
orbitofrontal cortex
links sensory and motor function
basal ganglia
pleasure centers of the brain
CENTER FOR DOPAMINE MODULATION
venture tegmental area (VTA)
dopaminergic VTA axons extends to the ____ in the ventral stratum of the basal ganglia
NAcc
some behaviors are rewarded & others are not (or may be punished) change in pop behavior
change in pop behavior
relevant to survival – assuring that needs are met, and dangers are avoided
operant conditioning + instrumental conditioning and reinforcement learning
adds desirable stimulus, do it more
- study (R) –> earn a good grade (O)
study more
pos reinforcement
subtracts a undesirable stimulus, do it more
ex. take aspirin (R) -> headache ends (O) = take aspirin for pain more often
negative reinforcement
adds undesirable stimulus, do it less
ex. arrive late (R) –> point penalty applied (O) = arrive late less often
pos punishments
subtracts desirable stimulus, do it less
ex. curse at ref –> ejected from game (O)= curse at reds less often
negative punishment
if normal ___ ___ program is blocked, animals will use other methods to achieve same ends
rote motor
not a single behavior, but a class of behaviors producing an EFFECT. some cognitive psychologists would call it more abstract goal or intention
R is a “behavioral unit”
operant conditioning is based on _____ (if R –> O)
contingencies
tells us which contingencies are in effect
- if S, R –> O
- if not S, R does nothing or a different O
- crucial ensuring that we select behaviors that are appropriate o the situation
discriminative stimulus
the ___ learned is an overall goal/intention, not just a simple behavior
response
outcomes: food delivery, shock
behaviors: rate of level pressing
context: light that signals box is “on”
skinner box
low spontaneous rate of R
before training
contingency is introduced :if s,r –> o
training
animal discovers contingency rate of R increases
acquisition
contingency is eliminated (R –> __) ; rate of R decreases
extinction
R has a ___ initial rate; animal must discover the contingency via exploration
low
requires integrated activity in many brain areas
reinforcement learning
____ ____ of the basal ganglia seems to-lay role in linking the stimulus with the response (S–R)
dorsal striatum
plays a role in linking response with outcomes (R–>O)
orbitofrontal cortex
linking sensory and motor function
basal ganglia
within basal ganglia, the ___ ___ caudate nucleus and putamen) plus importune role in s –r learning
dorsal striatum
____ ____ produce problems of reinforcement learning
- also links discriminative stimuli to behavioral contingencies (S–>R relationships)
dorsal striatum
directly activates the brain’s reinforcement system
electrical brain stimulation (intracranial self stimulation, ICC)
longer delay btw R & O –> ____ effect of punishment
weaker
shorter delay btw R & O –> ____ effect of punishment
stronger
signals contingency is in effect
discriminative stimulus
contingency should always be in effect
effective punishment
animal may learn discriminative stimuli that help it avoid punishment
- can produce fear,anxiety, Rage that impair behavior
- can produce generalized behavior disruption
- can produce aggression
circumvention
therapist disagree over the utility of ___ in therapy and child rearing
punishment
dopaminergic neurons in the ___ project directly to the dorsal striatum
SNc
dopamine gives reinforcers their hedonic qualities — the “Liking” aspect of reinforcement
- DA disrupted animals continue to show hedonic rxns, arguing against anhedonia hypothesis
*NOT SUPPORTED BY DATA
anhedonia hypothesis
dopamine motivates learners to work for reinforcement, the “wanting” aspect of reinforcement”
incentive salience hypothesis
dopamine blocker ____ disrupts dopamine, so food no longer good and is no longer reinforcing
pimozide
DA neurons in a monkey’s ____ responds ____ after unexpected rewards
midbrain
if light stimulus occurs before expected for reward, DA neurons ____ activation after light, but not after food
increase
DA neurons ___ activity after an expected reward does NOT OCCUR (Omission)
decrease
DA may be involved in adjusting the strength of S–R–C association through ___ ____ ___
reward prediction error
____ ____ may mediate “liking”
endogenous opioids
whether reinforcement is expected and codes for errors in reward prediction that drive reinforcement learning
reward prediction hypothesis
___ may be involved in “wanting”
DA
___ ___ may be involved in “liking”
endogenous opioids
gain knowledge of results during training – indication of how well you are preforming
- frequent ___ is helpful initially but doesn’t lead to high long term performance
- infrequent ____ causes a slower start, but usually leads to better long term performance
feedback
why deep expertise is so rare
- extreme practice required to gain advanced skill
power law
practice should be SPACED OUT rather than crammed/massed together
spacing
additional techniques for improvement
- additional power curve of performance increase
power law of learning
output/motor command adjust to reduce signed error, computed sensory feedback of mismatch btw output and target
CC: error related feedback
successful responses are reinforced
RL: reward-related feedback
single skill under identical conditions
constant practice
set of skills different conditions and contexts
- perform better during training and after training
variable practice
- able to play without looking at controller but need to focus; cannot get distracted otherwise more mistakes
associative stage
skills do/don’t decay with non-use?
do
- rapid decay when practice is first terminated
- slower decay as time progresses
- savings in relearning when practice is resumed
skill decay
interference from newly learned habits may impair ____ retention
skill
generalization of a skill from one context to another
transfer
- more shared (identical elements(, the better skills will transfer
identical elements theory
- initial learning of a new skill often tends to require ____ attention to verbal guidance or a set of rules
conscious
with practice, in skill learning, steps become ____ as attention to the verbalizable rules become unimportant
motor program / automatic
sequence of movements that an organism perform automatically
motor program
- performance is based on verbalized rules
- ex. using written instructions to set up a tent
cognitive stage
- actions become stereotyped
- setting up a tent in a fixed sequence without instructions
associative stage
- movements seem automatic
- setting up a tent while having discussion about politics
autonomous stage
skill learning depends on what 3 brain regions
cerebral cortex; cerebellum; basal ganglia
- flexible, consciously accessible, easy to verbalize
MTL + HIPPOCAMPUS
- bilateral MTL damage can no longer form new declarative memories, but can learn new skills
declarative memory
- rigid, not always consciously accessible, hard to verbalize
BASAL GANGLIA + CEREBELLUM + CEREBRAL CORTEX
- basal ganglia damage are impaired at skill learning
procedural/ skill memory
- interconnected clusters of neurons (glia) that sit at the BASE of the forebrain
- collects input form throughout the cortex
- sends output to:
— thalamus –> cortex
— brainstem
basal ganglia and skill learning
- skill training produces task-relevant coding of neural activity in the ____ ____
- increasing activity as skills are learned
basal ganglia
damage to the MTL impairs ?
declarative memory
basal ganglia damage impairs ?
skill learning
training expands ___space representing skill movements
- this expansion is accompanied by increased gray matter volume
- disuse shrinks ____ space representing skill movements
- neurons in motor cortical areas begin to fire at initiation of learned sequential motor program
cortical
skill practice expands the amount of ___ space dedicated to representing soil-related sensations and movements
cortical
violinist have expanded ______ representations of their playing hand, but not their bow hand
sensorimotor
performance improvement correlates with ____ of motor cortex activation
power law
neurons in monkey ____ in the frontal cortex fire selectively at initiation of specific learned sequences of movements
- coding for sequential motor program in cortex
supplementary motor area (SMA)
- takes in inputs from the spinal cord and cortical motor systems
- lesions impair performance of motor sequences and eye blink conditioning
cerebellum
- important in learning precisely timed movements sequences ex. acrobatics, dancing
cerebellum
acrobatic animals tend to have a larger ____
cerebellum
complicated motor tasks ex. balancing tightropes in obstacle course developed more _____ ____ than rats merely running an exercise wheel
cerebellar synapses
those with cerebellar lesions have difficulty ___ the opening of fingers during ball throwing
timing
copying a figure using a mirror image of the figure and one’s hand
- requires MOTOR ADAPTATION
mirror tracing
adapt to environmental perturbation of new sensorimotor relationship
motor adaption
human pts with __ damage have poorer initial skill + final performance, but EQUIVALENT LEARNING RATE#
cerebellar
on transfer test, cerebellar pts are even more impaired, demonstrating more/less to mirror tracing of different and more complex figures
less
mirror reading effects:
left cerebellum activity ____, right cerebellum activity _____
decreases; increases
motor control is the result of interactions btw what?
cerebellum, cortex, and basal ganglia
transforms perception into motion action
cerebral cortex
mediate cortical processing
basal ganglia + cerebellum
skill learning results from ____ ____ in these circuits
synaptic plasticity
lesions selectively impair skill learning
- development of task-related firing
basal ganglia
expanding gray matter and representational space for task-related sensory and motor areas
cerebral cortex
critical learning and performing movement sequences that require precise timing, coordination and/or motor adaptions
cerebellum
poor coordination of purposeful, skilled movements; common from head trauma or stroke
- cortical damage leads to selective ___
apraxia
damage causes difficult imitation actions
left parietal damage
Cannot pantomime actions with 2 hands
frontal damage
hinders control and execution of skills
cortical damage
cortical function can be temporarily altered using _____
- can temp boost function of stimulated cortex or temporarily disable it (virtual lesion)
TMS
- current induced directly in Brian without affecting sensory neurons in the scalp
TMS
temp mimic many selective apraxia observed in clinical population
TMS
increasing muscular rigidity, difficult initiating voluntary movements
- degeneration of dompaningeric brainstem neurons that modulate BG + cerebral cortex activity, resulting in DECREASED DOPAMINE
- difficulty learning skills such as serial reaction time, serial interception, rotary pursuit, and weather prediction tasks
parkinson’s disease
- levodopa
- deep brain stimulation
- stream cells + neurotrophic factors
parkinson’s disease symptoms temp
- inherited; causes damage to neurons in the brain (especially in BG + cerebral cortex)
- facial twitching + progressive shaking of body parts
- slow loss of motor abilities
- psychological problems (mood disorders, hyper sexuality, psychosis)
- difficulty with mirror reading and weather prediction tasks
Huntington’s Disease
encodes the protein huntingtin
HTT gene
- cortical region damage
- difficulty producing purposeful movements
- interferes with skill control and execution
- study effects of cortical disruption on skill memory and performance (TMS)
Apraxia
- disruptions in normal BG functioning due to Sec degeneration
- muscle tremors and rigidity
- deficit in skill learning
- dopamine replacement therapy (LEVODOPA) + deep brain stimulation (delivers an electrical current to the basal ganglia-cortical loop)
Parkinson’s disease
- inherited with deterministic genetic marker
- gradual damage to neurons throughout brain (especially in BG + cerebral cortex)
- deficit in skill learning
Huntington’s disease
- explicit conscious control; rehearsed verbally
cognitive stage
stereotyped movements learned, still exerting conscious control
associative stage
little or no cognitive effort
autonomous stage
with conscious awareness/effort
implicit learning
adept at implicit ____learning
skill
- given a cue & pressed an associated button
- random order, repeating sequence
- very fast at reacting to cues of the repeating sequences
- none of the participants were aware of the sequence
implicit skill learning
- press one of 4 keys to interest scrolling visual cues
- random order, repeating sequence
- more accurate at intercepting cues of the repeating sequence
- few participants were aware of the sequence
serial interception sequence learning task
inability to form new declarative memories
anterograde amnesia
amnesiac pt such as HM can still _____ learn new skills
- do not consciously call their practice sessions
implicitly
new situation shares many elements with the practice context
identical elements theory
skill learning builds motor programs (automatic sequences of motor activity in 3 stages
cognitive, associative, autonomous
