Week 2 Flashcards
What are the two types of associative learning?
Classical conditioning
Operant conditioning
Classical Conditioning
Ivan Pavlov
• Learning via association
Operant Conditioning
B.F. Skinner
• Learning via reinforcement
Free Energy Principle
Biological systems must maintain their states
despite a constantly changing environment (both
external and internal)
• The physiological and sensory states in which an
organism can be is limited - low entropy
• Entropy = surprise. a “fish out of water” has high entropy
• Biological agents must minimize the long-term
average of surprise to keep sensory entropy low.
Action or control signals, external states, sensations, internal states
Free-energy bound on surprise- action minimises prediction errors or perception optimises prediction
Free energy principle summed up
Minimising surprise, or….
maximising the sensory evidence for an agent’s existence (a model of its world).
A predictive/Bayesian brain
Classical (Pavlovian) Conditioning
Russian physiologist
Studied the digestive system.
Salivation.
Dogs that had been through the testing procedure several times: salivation observed before food was placed in their mouth
Learning. New stimuli.
…accidentally discovered
classical conditioning!
Pavlov’s salivation experiments
When food was placed in the animal’s mouth, the natural reflexive response of salivation occurred & was measured.
A stimulus elicits an innate response
Other stimuli don’t elicit that response
However dogs that had been through the testing procedure a few times would being to salivate before food was put into their mouths. therefore the dog has learned that footsteps, clanging of food dish, bell, whistle by the experimenter means food therefore salivating
4 elements of classical conditioning
- Food (US) -> Salivation (UR)
- Whistle (Neutral Stimulus) -> No salivation (No conditioned response)
- Whistle + food -> Salivation (UR)
- Whistle (CS)-> Salivation (CR)
Definitions of the 4 elements of classical conditioning
Unconditioned stimulus (US)- A stimulus that elicits an unlearned response Unconditioned response (UR)- The learned response to a US Conditioned stimulus (CS)- A stimulus to which an organism must learn to respond Conditioned Response (CR)- The response to a CS (which is learned)
Steps of conditioning
Before- US-> UR
During- CS+US-> UR
After- CS-> CR
Clinical applications for classical conditioning
Acquisition of fears, phobias, and other maladaptive behaviours.
Treatment of these same fears, phobias, and other maladaptive behaviours.
Conditioned fear- Watson & Rayner
Little Albert
Acquisition of emotional responses- loud noise and mouse
generalised fear for rabbits and santa
The point was that you can train people with certain behavious
A typical classical conditioning experiment
-Stage 1: Habituation – CS presented alone
• Stage 2: Acquisition – CS presented along with US
• Stage 3: Extinction – CS presented alone again
- Throughout we measure a response (UR/CR)
A typical experiment: Habituation
US causes UR
Then CS does not cause CR
A typical experiment: Acquisition
CS does not cause CR
US causes UR
Repeating eventually CS causes CR and US causes UR
The strength of CR increases with more trials
A typical experiment: Extinction
CS causes CR but over time CS does not cause CR
Two factors influence the acquisition curve:
Intensity of the US (more intense, more rapid learning)
– Order and timing (the CS coming before the US is better)
Timing: Delay Conditioning-Short
There is a short interstimulus interval (ISI) of the CS before the US
Timing: Delay Conditioning-Long
There is a long interstimulus interval (ISI) of the CS before the US
Timing: Trace conditioning
There is a trace interval (a gap) after the CS and before the US
Timing: Simultaneous conditioning
CS and the US are at the same time
Timing: Backward Conditioning
US is before the CS
Back to Delay conditioning: Is there an optimal ISI?
Interstimulus interval of CS before the US
Two Types of Pavlovian Conditioning
• Excitatory conditioning
– CS predicts the occurrence of US
– example: if ‘A’ is a bell… A-US, A-US, A-US
• Inhibitory conditioning
– CS predicts absence of US
– example: if ‘B’ is a light… A-US, A-US, AB, A-US, AB
• here B predicts the absence of US
Inhibitory conditioning
• A-US, A-US, A-US —> A leads to a CR (excitatory
conditioning)
• A-US, AB-nothing, A-US, AB-nothing –> B?
• The animal learns that B predicts the absence of US and therefore doesn’t make a CR (inhibitory conditioning)
• The problem: How do we know that animal learned something about B summation test and retardation tests
An inhibitor must pass both
Retardation test
• First inhibitory conditioning takes place
– A-US, AB-nothing, A-US, AB-nothing –>
– B becomes inhibitor ‘I’
• To test it - train an inhibitor ‘I’ and a neutral stimulus ‘N’ to become excitatory
– I-US, I-US, I-US
– N-US, N-US, N-US
– Slower learning to inhibitor: I < N
Summation test
First inhibitory conditioning takes place
– A-US, AB-nothing, A-US, AB-nothing –>
– B becomes inhibitor ‘I’
• To test it - Present:
– A new excitatory CS alone: ‘N’
– A new excitatory CS + the inhibitor: N+I
N+I < N
The combo should evoke a
WEAKER CR
What happens during extinction?
Acquisition (CS+US) increases the strength of the CR but Extinction is when CS is presented alone
Extinction is not simply earsing whatever association was formed during acquistion
Spontaneous recovery
Reintroduce the CS after a “break” and the CR appears
Renewal
§ Renewal: when extinction is context specific
- Acquisition in context X
- Extinction in context Y
- Present CS in context X: CR
Reinstatement
Reinstatement (reminder effect)
- present US alone after extinction
- then present CS = CR
The hidden assumptions of classical conditioning
e.g.,
• Any two stimuli can be paired together (equipotentiality)
• The more two stimuli are paired, the stronger the individual will associate them (contiguity)
• Conditioning changes trial to trial in a regular way (contingency)
…. Blocking and Superconditioning show us these are wrong
Blocking- rat example
Kamin (1968) • Rats divided into ‘Blocking’ and ‘Control’ groups. • In the control group: – rats saw both a light and heard a noise and then got shocked – Repeat until rats developed a CR • Kamin then tested whether the rat reacted to just the light by trying to avoid being shocked (it did). • In the blocking group: – rats heard a noise and then got shocked until they developed a CR – rats then heard the noise and saw the light and then got shocked • Kamin then tested whether the rat reacted to the light by trying to avoid being shocked. • It didn’t. • Rats in both the blocking and control groups had seen the light and then been shocked exactly as many times • However, the rats reacted differently • This is called the Blocking Effect
The blocking effect and hidden assumptions
Any two stimuli can be paired (equipotentiality)
The more two stimuli are paired, the stronger the individual will associate them (contiguity)
Paring the light with the shock didn’t lead to conditioning
The control group and the blocking group had the noise+light/shock pair the exact amount of times
Superconditioning- rat example
• The opposite of blocking • (Rescorla, 1971) • some rats were played a tone in the absence of a shock – tone became cue of safety (Inhibitor) • Later rats were presented the tone with a light followed by a shock • Finally rats were presented the light alone – rats in this group showed stronger conditioning to the light than rats in a control group
Blocking defined
• Blocking - When a neutral stimulus and an excitatory stimulus together are paired
with the US
• The learner does not
form an association between the neutral stimulus and the US
Superconditioning defined
Superconditioning – When a neutral stimulus and an inhibitory stimulus together are paired with the US • The learner forms a stronger association between the neutral stimulus and the US
