8 - Extinction & stimulus control

Question 1

What is extinction?

Answer 1

• The non-reinforcement of a previously reinforced response (procedure), the result of which is a decrease in the strength of that response (process)
  → the procedure here being no longer providing food at the lever press
  → the process here being the lever pressing decreasing
  → the process of extinction is the result of decrease, while the procedure is the non-reinforcement
• Extinction is not forgetting
  → Extinction is not simply a process of unlearning the conditioning
  → Extinction involves learning something new about the CS
  → Extinction involves learning that the operant response no longer predicts reward

Question 2

If lever pressing ceases entirely, the response is said to have been ___.

Answer 2

Extinguished

Question 3

If lever pressing has not yet ceased entirely, then the response has been only ___ ___.

Answer 3

Partially extinguished

Question 4

What is the critical first step in extinguishing a behaviour? Give an example.

Answer 4

• Determining the effective reinforcer that is maintaining a behaviour
  → Child wines (R) -> Gets candy (Sr)
  → Child wines (R) -> Attention from parent (Sr)
  → it might not be necessarily getting candy which is reinforcing the child’s whining, but rather the accompanying attention from the parent
  → if it’s the latter, then withholding the candy might have little or no effect on the behaviour (in this case, the parents attention is the reinforcer, rather than withholding the candy)
  → another possibility is that whining is being maintained by both the candy AND attention, in which case withholding the candy would only partially extinguish the behaviour
  → this is why determining the effective reinforcer is a critical first step
  → Child wines (R) -> Gets no candy + Gets no attention (no reinforcer) -> no more response

Question 5

When an extinction procedure is implemented, it is often accompanied by certain side effects that seem a lot like the 5 stages of grief; what are these side effects?

Answer 5

→ extinction burst (denial)
→ increase in variability (denial)
→ emotional behavior/frustration (anger)
→ aggression (anger)
→ resurgence (bargaining)
→ depression (depression)
→ acceptance ? (not a side effect of extinction)

Question 6

What is extinction burst?

Answer 6

• A temporary increase in the frequency and intensity of responding when extinction is first implemented
  → Rat will press the bar rapidly when food is no longer delivered
  → The child may scream with increased enthusiasm when not rewarded.
  → Repeatedly hit the ’open door’ button when the elevator won’t move.
  → You push ‘on’ button repeatedly when your phone won’t respond

Question 7

What is an increase in variability as a side effect of extinction?

Answer 7

• Trying a variety of methods for attaining the reward
  → Rat will try pressing the side of the bar, or using the left paw instead of the bar rapidly when food is no longer delivered
  → You rapidly push all the apps on your phone when it won’t respond
  → You hit all the buttons in the elevator when it won’t move

Question 8

What is emotion behaviour/frustration as a side effect of extinction?

Answer 8

• We typically refer to these behaviors as frustration
• Examples:
  → Agitated behavior of a the rat
  → Screaming in the stuck elevator
  → Yelling at your phone

Question 9

What is aggression as a side effect to extinction?

Answer 9

• An emotional response to extinction.
• Examples:
  → A rat whose lever pressing is placed on extinction may attack another rat

Question 10

What is resurgence as a side effect of extinction?

Answer 10

• The reappearance during extinction of other behaviors that had once been effective in obtaining reinforcement
  → A rat may begin digging in the ground because it has previously found food in its bedding
  → You may try holding the power switch and the ‘on’ button

Question 11

What is depression as a side effect to extinction?

Answer 11

• Low activity is a common symptom of depression.
• Example:
  → The hungry rat withdraws into the corner of the cage.
  → The guy with the broken iPhone withdraws into the corner of his cubicle
• often associated with the loss of reinforcement, so if someone dies that is important to you and was a major source of reinforcement, you are essentially experiencing extinction and will become depressed over time

Question 12

What are some problems with extinction side effects?

Answer 12

• Side effects of extinction impede successful implementation of extinction
• Side effects can be inadvertently strengthened if one suddenly gives in and provides the subject with the sought-after reinforcer
  → p.ex: if a parent suddenly gives in and buys the candy after their kid throws a temper tantrum, then the mother has essentially reinforced the wrong behaviour
  → parents can inadvertently strengthen temper tantrums
• Kids who think it’s acceptable to throw temper tantrums become adults who think it’s acceptable to throw temper tantrums

Question 13

What is resistance to extinction?

Answer 13

• The extent to which responding persists after an extinction procedure has been implemented
• A response that is very persistent is said to have high resistance to extinction
  → p.ex: Begging continues for over an hour = high resistance
• A response that disappears quickly is said to have low resistance to extinction
  → p.ex: Begging stops after 5 mins = low resistance

Question 14

Persistence of responding may be influenced by the presence of a ___ ___.

Answer 14

Discriminative stimulus
→ Kid (Sd): Dog begs for food (R) -> Begging will be persistent (high)
→ Mother (Sdp): Dog begs for food (R) -> Begging will extinguish (low)

Question 15

In relation to resistance to extinction, when trying to modify behaviour, ___ in the ___ of behaviour is important.

Answer 15

• Consistency in the outcome
  → Kid (Sd): Dog begs for food (R) -> Gets table scraps (Sr)
  → Mother (Sdp): Dog begs for food (R) -> Gets yelled at (Sp)
  → p.ex: you don’t say no to a kid for a few minutes and then give in, you stay consistent

Question 16

Resistance to extinction can be affected by a number of factors, what are they?

Answer 16

• Schedules of reinforcement
• History of reinforcement
• Magnitude of reinforcer
• Degree of deprivation
• Previous experience with extinction
• Distinctive signal for extinction

Question 17

Explain how schedules of reinforcement can influence extinction.

Answer 17

• Scheduled reinforcement is the most important factor influencing resistance to extinction
  → this is indicated by the partial reinforcement effect
• Partial reinforcement effect
  → behavior maintained on an intermittent schedule will extinguish more slowly than behavior maintained on a continuous schedule
  → Resistance to extinction is particularly strong following a variable ratio schedule
  → the randomness in the schedule itself is providing strong resistance to extinction

Question 18

Explain how the history of reinforcement can influence extinction.

Answer 18

• The more reinforcers an individual has received for a behavior, the greater the resistance to extinction
  → p.ex: 100 reinforced trials vs. 3 reinforced trials
• Easier to extinguish an unwanted behavior when it first becomes evident
  → Kid (Sd): Dog begs for food (R) -> Gets table scraps (Sr)
  → Dog begs for food (R) -> Begging will be persistent (high resistance)
  → p.ex: If timmy whines for candy for a long time (several weeks) and they’ve gone through 100 reinforcers, it’ll be harder for him to quit (extinguish), than Alex who just recently started watching him and imitating this behaviour

Question 19

Explain how the magnitude of the reinforcer can influence extinction.

Answer 19

• Large-magnitude reinforcers result in greater resistance to extinction than small-magnitude reinforcers
  → p.ex: small bag of food may not orient dogs attention as quickly as a large bag of food

Question 20

Explain how the degree of deprivation can influence extinction.

Answer 20

• The greater the level of deprivation, the greater the resistance to extinction
  → A rat that is only slightly hungry will cease lever pressing more quickly than a rat that is very hungry
• The ‘motivating operation’ plays a role in the susceptibility to extinction

Question 21

Explain how previous experience with extinction can influence future extinctions.

Answer 21

→ When sessions of extinctions are alternated with sessions of reinforcement, the greater the number of prior exposures to extinction, the quicker the behavior will extinguish during subsequent exposures

Question 22

Explain how a distinctive signal for extinction can influence extinction.

Answer 22

• Extinction facilitated when there is a distinctive stimulus/discriminative stimulus (SD) that signals the onset of extinction
  → Mother (Sdp): Dog begs for food (R) -> Begging will extinguish (low resistance)

Question 23

What is differential reinforcement of other behaviour (DRO)?

Answer 23

• Reinforcement of any behavior other than the target behavior that is being extinguished
• Extinction can be greatly facilitated by both extinguishing the target behavior and reinforcing the occurrence of a replacement behavior

Question 24

What is differential reinforcement of incompatible behaviour (DRI)?

Answer 24

• Reinforcing a behavior that is specifically incompatible with the target behavior
• With DRO & DRI, there is no deprivation of reinforcement and thus the likelihood of side effects is greatly reduced
  → p.ex: paying attention to a child if they’re doing anything other than fighting with their sister is a DRO procedure, and paying attention when they’re acting really friendly with their sister is a DRI procedure

Question 25

The presence of a discriminative stimulus reliably affects the ___ of the behaviour.

Answer 25

Probability
→ the DS does not automatically elicit the behaviour, it merely signals the availability of reinforcement, thereby increasing the probability of behaviour
- Example:
→ A 2,000-Hz tone signals that lever pressing will lead to food
→ 2000-Hz tone (Sd): Lever press (R) -> Food (Sr)
→ the rat will thus learn to press the lever when hearing the tone

Question 26

Behaviors (responses) that are likely to occur is the presence of a Sd are under ___ ___.

Answer 26

Stimulus control

Question 27

How can apply the use of stimulus control?

Answer 27

• Stimulus control can be used to eliminate problem behaviors
• Putting a behavior ‘on cue’ minimizes the likelihood of the behavior occurring at other inappropriate times (in the absence of the cue)
• Say you want to minimize the time you spend on Facebook.
  → A push notification (cue) tells you you have a message.
  → Only check your Facebook account when you get a notification
• Stimulus control can be used to create effective study environments
  → where you study can allow you to focus more or less
  → it can be a stimulus control in the sense that every time you’re in that environment, it allows you to be more productive
  → Skinner (1987) recommends establishing a particular setting that is used only for studying.
  → Your desk becomes the cue for ‘studying’

Question 28

What is stimulus generalization?

Answer 28

• The tendency for an operant response to be emitted in the presence of a stimulus that is similar to the original SD
• The more similar the stimulus, the stronger the response
  → the rat will still lever press to lower tones

Question 29

What is the generalization gradient?

Answer 29

• The strength of responding in the presence of stimuli similar to the SD varies along a continuum
• A flat gradient indicates strong generalization (left)
  → indicates that responding drops gradually as the stimuli becomes increasingly different of the Sd
• A steep gradient indicates weak generalization (right)
  → indicates that the rate of responding drops sharply as the stimuli becomes increasingly different from the Sd

Question 30

What is stimulus discrimination?

Answer 30

→ The tendency for an operant response to be emitted more in the presence of one stimulus than another.
→ A flat gradient indicates strong generalization/weak discrimination.
→ A steep gradient indicates weak generalization/strong discrimination

Question 31

What is discrimination training?

Answer 31

• Reinforcement of responding in the presence of one stimulus (the SD) and not another stimulus (S△)
• A discriminative stimulus for extinction (S△) is a stimulus that signals the absence of reinforcement
• Example:
  → Whenever the 2,000-Hz tone sounds, a lever press produces food; whenever the 1200-Hz tone sounds, a lever press does not produce food

Question 32

Explain the peak shift effect.

Answer 32

• The peak of a generalization gradient following discrimination training will shift from the SD to a stimulus that is further removed from the SΔ
  → this is just an exception to the general principle that the strongest response occurs in the presence of the Sd
• Example: first train rat to press lever in presence of 2000 htz tone → then conduct test for generalization across range of tones varying in pitch (1200-2800)
  → prior to discrimination training: figure to the left
  → in bottom right figure: then submit rat to discrimination lever procedure: reinforce lever pressing in presence of 2000 htz tone and not 1200 htz tone
  → then again test for generalization across tones (this time rat only responds in the presence of 2000 htz tone, and not 1200 htz
  → prior to discrimination training, gradient/slope relatively flat → but once 1200 htz tone established delta, the strongest response occurs in the presence of stimulus further removed from delta (as demonstrated by gradient to the right)

Question 33

What are multiple schedules ?

Answer 33

• Two or more independent schedules presented in sequence, each resulting in reinforcement and each having a distinctive SD
• Example:
  → A pigeon is presented with a VR 20 schedule on a green key, followed by an FI 10-sec schedule on a red key
  → Green key (Sd): Peck (R) -> Food/Red key (Sr / Sd): Peck (R) -> Food (Sr)
  → p.ex: in order to do well in a chem exam, you’ll study to get an A

Question 34

What is a chained schedule?

Answer 34

• A type of multiple schedule of behavioural contrast
• A sequence of two or more simple schedules, each of which has its own SD and the last of which results in a terminal reinforcer (delivered only at the end, contrary to the multiple schedule)
• Must be completed in a particular order
• Example:
  → A pigeon is presented with a VR 20 schedule on a green key, followed by an FI 10-sec schedule on a red key, which then leads to the terminal reinforcer of food
  → Green key (Sd): Peck (R) -> Red key (Sr / Sd): Peck (R) -> Food (Sr)

Question 35

What is the difference between multiple schedules and chained schedules?

Answer 35

• Multiple schedules:
  → 2 or more schedules completed in a sequence
  → each schedule has a reinforcer
• Chained schedule:
  → 2 or more schedules completed in a sequence
  → There is a terminal reinforcer

Question 36

Define behavioural contrast.

Answer 36

• A change in the rate of reinforcement on one component of a multiple schedule produces an opposite change in the rate of response on another component
• As the rate of reinforcement on one component changes in one direction, the rate of response on the other component changes in the other direction

Question 37

In a behavioural contrast, what is a negative contrast effect?

Answer 37

• An increase in the rate of reinforcement on one component produces a decrease in the rate of response on the other component
• Example:
  → A pigeon is on a multiple VR 60-sec VR 60-sec schedule and the VR 60-sec component on the red key is changed to VR 30-sec.
  → What will happen to the green key responding?
  → With more reinforcement now available on the red key, the pigeon will decrease its rate of response on the green key

Question 38

In a behavioural contrast, what is a positive contrast effect?

Answer 38

• A decrease in the rate of reinforcement on one component produces an increase in the rate of response on the other component.
• Example:
  → A pigeon is on a multiple VR 60-sec VR 60-sec schedule and the VR 60-sec component on the red key is changed to VR 120-sec
  → What will happen to the green key responding?
  → With less reinforcement now available on the red key, the pigeon will increase its rate of response on the green key

Question 39

In the contrast effect, as the one alternative suddenly becomes ___, the remaining alternative, which is still reinforcing, is viewed as relatively more ___.

Answer 39

Punishing; attractive

Question 40

What is anticipatory contrast?

Answer 40

• The rate of response varies inversely with an upcoming (“anticipated”) change in the rate of reinforcement
• Examples:
  → Pigeons increased their rate of responding for reinforcement when presented with a discriminative stimulus for extinction
  → Things we are about to lose often seem to increase in value
  → p.ex: lindsay views her relationship with Bryce as dull, but she learns that he might be starting to like someone else, so she starts seeing the relationship in a better light

Question 41

What is spontaneous recovery?

Answer 41

• Spontaneous recovery is the reappearance of an extinguished response following a rest period after extinction

Question 42

Errorless discrimination training reduces many of the ___ ___ and ___ associated with discrimination training.

Answer 42

Sife effects; errors