Week 5 Flashcards
Classical vs Operant Conditioning
Classical Conditioning • Ivan Pavlov • Learning via association • (Lectures 2 + 3) Operant Conditioning • B.F. Skinner • Learning via reinforcement • (Lectures 4 + 5)
Theories of reinforcement
- The role of response and reinforcement
- Skinner
- Drive reduction
- Behaviour regulation (Premack)
Learning by doing?
• Thorndike thought that a response had to be performed and a consequence experienced for operant learning to occur • However, Tolman showed that this is not so – Responses: e.g. learning without doing (e.g. rats running a maze vs rats transported through maze, learned the same way) – Consequences: e.g. rats acquire a cognitive map of their environment even if not rewarded. So-called latent learning (Tolman & Honzik, 1930)
Expectations about the reinforcer
• Thorndike thought that learning involved the
pairing of Stimulus and Response – without
inclusion of the reinforcer
• Tolman argued that the reinforcer becomes part of
associative network (stimulus, response and
reinforcement)
– Animal develops expectation
• E.g. train monkey to perform action that leads to
banana, and then replace reward with lettuce ->
monkey surprised and frustrated
Experiment by DeWaal & Brosnan
Monkeys reject unequal pay
Theories of reinforcement:
Skinner’s operational definition
- Reinforcer increases rate of behaviour
* Punisher decreases rate of behaviour
Theories of reinforcement:
Drive reduction theory
Hull & Spence (1940s)
• 1st theory of motivation
• If homeostasis is disrupted drive is observed
– Primary drives are innate (e.g. thirst, hunger)
• Drive = unpleasant state that the animal wants to
reduce
• Drive reduction of physiological needs = negative
reinforcer & major cause of learning
But… not all reinforcers reduce a biological
drive
• Secondary reinforcers (e.g. money)
– Money can only indirectly reduce drive
– Bridging (previous lecture)
• Novel Stimuli – e.g. Sensation seeking
– Some reinforcement comes from raised
stimulation
– wild rats, mice and shrews elect to run
in wheels
– Working to get access to a window
• Pleasure seeking
• e.g. Intra-cranial reinforcers
Olds & Milner (1954)
Electrical stimulation as positive reinforcer
Intra-cranial Reinforcers
Rise of the ‘ratbots’?
- Talwar et al. (2002)
* Nature, 417, 37-38
What if…
• What if stimuli aren’t reinforcers – E.g., water isn’t a reinforcer • But behaviours are reinforcers – E.g., DRINKING is a reinforcer • Behavioural homoeostasis (not food, but eating is reinforcing) • “Bliss point” of behaviour
Theories of reinforcement:
Behaviour regulation
• Premack’s principle (1965)
– A high probability behaviour can reinforce a low
probability behaviour
▪ We have a hierarchy of behaviours arranged
according to response probabilities
(preferences)
The Premack Principle
More probable behaviours will reinforce less probable behaviours
Using Premack’s Principle
Brown, Spencer, & Swift (2002) saw a 7-year-old boy, who refused to eat all but a few specific foods • Low probability behaviour: eating new foods • High probability behaviour: eating favourite foods • At meal-times, parents told him if he ate a small amount of new food, he could have his favourites • Boy gradually began to eat his greens!
Avoidance vs. Escape
• Remember that punishment ideally involves no
possibility of avoidance/escape
• But, what if the animal has some control?
• Escape learning–emit a response that
terminates an aversive consequence (negative
reinforcement)
• Avoidance learning – emit a response to
prevent the occurrence of an aversive
consequence altogether
– but how can this be examined in a lab?
Present CS (light dims) followed by US (shock)
Initial trials feature escape, then avoidance takes over
Learning About Avoidance Learning
▪ Helps us to understand anxiety behaviours
▪ Phobias
▪ Often never encounter aversive event again
▪ How are phobias maintained?
How to Reduce Avoidance Clinically
• In therapy: exposure training
– Flooding and response prevention
– Modelling of situation appropriate behaviour
Learned helplessness
Seligman, 1967
▪ Effect of unavoidable shock ▪ Yoked control experiment ▪ (1) Escapable shock: Avoidance learning ▪ (2) Inescapable shock: Yoked to the first group’s shocks ▪ (3) No shock ▪ Then train in shuttle box avoidance task
Learned helplessness: Effects
▪ Impairs subsequent learning (in more difficult task) ▪ Depression ▪ Reduced activity ▪ Reduced immune responses ▪ More ulcers (stress related)
Learned Helplessness
• Repeated exposure to punishment has long-term
effects
• Animals/people start to behave as if their
behaviour has no effect on what happens to them
• This is learned helplessness – they’ve learned that
they’re helpless
How To Combat Learned Helplessness
Place the subject in a situation where it cannot fail,
so it learns it has some control
– An initial experience of control often
‘immunises’ against learned helplessness
Learned helplessness- attributions
In humans our attributions can harm or protect us
– Internal vs external
• (because of me/not because of me)
– Stable vs unstable
• (a trait that I have/one off incident)
– Global vs specific
• (applies to all contexts/applies to this one context
• Depression promoting attributions:
– Internal, stable, global
• Depression reducing attributions:
– External, unstable, specific
Learned Helplessness In Uni Students
• Learned helplessness is typically worse if: 1. The person thinks everything is hopeless 2. The person thinks it’s their fault 3. The person sees the helplessness as longterm • A uni student who has learned helplessness might think that: 1. I’m no good at any of my classes (global) 2. Because I’m just not very smart (internal) 3. And I’ve just always been that way. (stable)
Behavioural Therapies
• Use conditioning principles • Aim to modify situation inappropriate behaviours 1. Functional Analysis 2. Use empirically validated approach to modify specific behaviour/condition
Functional analysis
• What is the problem behaviour, when does it
occur?
• Helps us to target our treatment
• A functional analysis tries to determine what
reinforcers are maintaining an undesirable
behaviour
• Involves monitoring the relationship between
stimuli, behaviour, and consequences.
• Is done on a case by case basis (N = 1)
• Divide complex behaviours into simple ones
(more manageable)
• Inappropriate behaviour same as ‘normal’
behaviour (non-medical model of pathology)
• Inappropriate behaviour: excess or deficit in a
certain situation
Functional Analysis Example
• E.g., A mentally disabled 10 year old boy who had
self-injurious behaviour (SIB),
• Watson et al. (1999) performed a functional
analysis and found
– that the SIB happened most often as ‘escape behaviour’
– the boy did it in order to stop doing things he didn’t like.
• Thus, the best way to stop the SIB was to get his
teacher to switch to a fun task if he did an unliked
task without SIB.
Functional Analysis - SORCK
• Kanfer & Phillips (1970) argued that the essential ‘behavioural equation’ for a functional analysis included: – ‘prior Stimulation’ – the ‘biological state of the Organism’, – the ‘Response repertoire’, – ‘Consequence – the ‘Contingency relationship’ (K) • S: what happened before the behaviour • O: the skills and state of the organism at the time • R: the behaviour • C: the consequence of the behaviour • K: the effect the consequences of the behaviour have on future behaviour
Operant Conditioning in Action
• Positive Parenting Programme (Triple P)
• Teaches parents to use operant conditioning
principles in child rearing
• Focuses on reinforcement of desired
behaviour (rather than punishment)
• Teaches other principles too – like chaining,
making consequences more effective etc.,
Advice from Triple P
E.g. Getting children ready for school in the morning – Teach children the behaviours they need – Set a routine (an order for doing them in) – Reward desired behaviour – Avoid nagging or hassling them – Turn it into a game • Uses principles like chaining and reinforcement
Behavioural Therapy: Throwing ‘thinking’
back in the mix
• CBT is a combination of cognitive therapy and behavioural therapy – ‘cognitive behavioural therapy’ • focus on ‘thinking errors’ and ‘core beliefs’, which is cognitive • but other techniques are based on operant conditioning and classical conditioning, • e.g., practice exercises and setting homework • especially in the treatment of anxiety (where behavioural therapy was originally quite successful)
Use in comparative psychology
• Conditioning often is the basis of lean alternative explanations that
need to be ruled out to demonstrate animal smarts
• e.g. Ravens plan for the future (Kabadayi & Osvath, 2017, Science)
• “Here, we show that ravens plan for events …—tool-use and
bartering—with delays of up to 17 hours.”
• birds picked target from among distractors
• However, simpler explanations not ruled out
• Before E1, the 5 ravens were trained that a tool/ token leads to reward
and that distractors did not
• Unsurprisingly, they subsequently tended to select the functional items
and could then get a reward later
• Even on first trial, though there was no rational ground to predict that the
reward situation would emerge 17 hours later
• More about rich vs lean interpretations of animal behaviour in the
lecture on comparative cognition
