Chapter 5: Operant Conditioning

Question 1

outcomes are ________ of an animal’s behaviour

Answer 1

consequence

Question 2

Why is instrumental behaviour emitted?

Answer 2

Because it is effective in producing a particular consequence

Question 3

Operant conditioning?

Answer 3

a response defined by the effect it produces in the environment (goal- directed)

Question 4

E. L. Thorndike

Answer 4

Puzzle box
– Hungry animal placed in box
– Food visible to animal
– Measured how long it took to escape the box on successive trials

Thorndike measured latency to get out of box on successive trials

Law of effect

Question 5

Law of effect

Answer 5

Responses in presence of stimulus followed by satisfying events
will strengthen the association between the S and R;
If response followed by annoying event, association is weakened

Question 6

Standard tasks for examining instrumental behaviour: Discrete trial procedure

Answer 6

– Instrumental response produced once per trial
– Each training trial ends with removal of the animal from the apparatus

ex. Operant Devices
Mazes
Inspired by observing animals in nature (burrows of rats)
measure: Running speed, Latency, Correct choices

Question 7

Standard tasks for examining instrumental behaviour: Free-operant procedure

Answer 7

– Animals remain in apparatus and can make many responses
– No intervention by the experimenter
– Developed by BF Skinner
– Need a unit to measure behaviour:
•Operant response: defined by the effect that the response produces on the environment
– Response rates are often the behavioural measure

Question 8

Standard tasks for examining instrumental behaviour: Magazine training

Answer 8

– How you establish the operant response
– Animal must be shown how to “use” the experimental set up
– You need a series of training steps
– Involves classical conditioning

Question 9

Examining instrumental behaviour

• Shaping

Answer 9

– Sequence of training steps
1) Light always occurs with
food delivery 
2) Light and tone occur
with food delivery 
3) Tone occurs with food
delivery
• Light does not occur
anymore

shaping takes advantage of response variability

Question 10

Burrhus Frederick Skinner

Answer 10

shaping a pigeon to turn

Question 11

Response Variability

Answer 11

Behaviour starts off highly variable
Successful variations are maintained;
unsuccessful variations are not

Question 12

appetitive stimulus

Answer 12

pleasant outcome

Question 13

aversive stimulus

Answer 13

unpleasant outcome

Question 14

instrumental responses can

Answer 14

result in a stimulus

turn off a stimulus

Question 15

Reinforcer

Answer 15

Reinforcer : An event that follows behaviour,

and that behaviour increases

Question 16

Punisher:

Answer 16

Punisher: An event that follows behaviour,

and that behaviour decreases

Question 17

Reinforcement:

Answer 17

increases responding

Question 18

Punishment

Answer 18

decreases responding

Question 19

positive

Answer 19

add something (appetitive or aversive)

Question 20

negative

Answer 20

remove something (appetitive or aversive)

Question 21

positive punishment

Answer 21

response-outcome contingency: response produces an aversive stimulus

result: decrease in response rate

Question 22

positive reinforcement

Answer 22

response-outcome contingency: response produces an appetitive stimulus
Result: incerase in response rate

Question 23

Negative reinforcement

Answer 23

response-outcome contingency: eliminates or prevents the occurrence of an aversive stimulus
result: increase in response rate

Question 24

Negatove punishment

Answer 24

omission training

response-outcome contingency: response eliminates or prevents the occurrence of an appetitive stimulus

result: decrease in response rate

Question 25

Omission training (DRO treatment) for Self-Injurious Behaviour

Answer 25

Reinforced with attention when not performing self-injurious behaviour

Question 26

Reinforcer vs. Punisher is all based on the __________

Answer 26

Reinforcer vs. Punisher is all based on the

target behaviour

Question 27

Instrumental Conditioning: The Players

Answer 27

1. Instrumental response
2. Outcome of response (reinforcer)
3. Response-reinforcer relation

Question 28

Instrumental response

Answer 28

• Thorndike “stamping in” of S-R association
• Skinner “reinforcement” or strengthening of
behaviour
• Language suggests that instrumental conditioning results in necessarily stereotyped behaviours
• Led to studies of Response Variability

Question 29

Response Variability

Method:

Answer 29

– Pigeons in operant chamber
– Peck two response keys a total of 8 times
– No restriction on distribution of pecks between the two keys
– BUT, pattern of left/right pecks on a given trial had to be different that on the previous 50 trials
– Only “novel” patterns reinforced

Summary:
• Response variability can be increased by
reinforcement
• In absence of reinforcing variability in responses, responding becomes more stereotyped

Question 30

Relevance or Belongingness

Answer 30

Relevance or Belongingness
• Remember bright-noisy/water Classical
Conditioning experiment with rats?
– (taste with sickness, audiovisual with pain)

• Belongingness - Proposed by Thorndike
– Evolutionary history makes certain responses belong
with certain reinforcers
– Pawing at door vs. yawning

Question 31

The outcome or reinforcer

Answer 31

• (1) Quality and quantity
• (2) Previous experience with other reinforcers
(shift in quality or quantity)
– Similar to R-W
• Larger than expected reinforcer (or US) supports excitatory conditioning
• Smaller than expected reinforcer (or US) supports inhibitory conditioning

Question 32

Manipulation of Quality vs. Quantity

Answer 32

• Independent groups of rats given flavoured water
reinforcement • More responses when reward is greater in quantity
and/or quality

Question 33

Response-reinforcer relation

Answer 33

Temporal relation

Causal relation

Question 34

Temporal relation

Answer 34

Time between response and
outcome
– Reinforcement is most effective if it happens
immediately after the target response

Question 35

Causal relation

Answer 35

The extent to which the response
is necessary and sufficient for the occurrence of the reinforcer
– Reinforcement is most effective if it only happens afterthe target response

Question 36

Effects of Temporal Contiguity

Answer 36

• Immediate reinforcement preferred
• Delays over 0.5 s can hinder performance
• Recent research suggests: can use delays up to 30 s

Question 37

Effects of Delayed Reinforcement

Answer 37

Delay between response and food reinforcement in rats

No learning in 64-sec delay condition

Question 38

How to overcome delay in reinforcement?

Answer 38

1. Conditioned (secondary) reinforcer

2. Marking procedure

Question 39

Conditioned (secondary) reinforcer

Answer 39

A stimulus that becomes an
effective reinforcer because of its association with a primary or unconditioned reinforcer (e.g., food)
helps overcome delay in reinforcement

Question 40

Marking procedure

Answer 40

Instrumental response is immediately followed by a distinctive event that makes the instrumental response more memorable

helps overcome delay in reinforcement

Question 41

Response–reinforcer contingency:

Answer 41

The relation of a response to a reinforcer
defined in terms of the probability of getting
reinforced for making the response as
compared to the probability of getting
reinforced in the absence of the response

Question 42

Skinner’s Superstition Experiment

Answer 42

Pigeons trained with temporal-contingent

reinforcement exhibit “superstitious” behaviours

Question 43

Behavioural Momentum

Answer 43

More than just response rates, reinforcers
strengthen the tendency to persist in
behaviour

Question 44

Reinforcers: primary vs. secondary

Answer 44

Primary: US
Secondary: CS (i.e. money)

Question 45

Reinforcers: intrinsic vs. extrinsic

Answer 45

inside

outside (i.e money)