8 - Extinction & stimulus control Flashcards
What is extinction?
- The non-reinforcement of a previously reinforced response (procedure), the result of which is a decrease in the strength of that response (process)
→ the procedure here being no longer providing food at the lever press
→ the process here being the lever pressing decreasing
→ the process of extinction is the result of decrease, while the procedure is the non-reinforcement - Extinction is not forgetting
→ Extinction is not simply a process of unlearning the conditioning
→ Extinction involves learning something new about the CS
→ Extinction involves learning that the operant response no longer predicts reward
If lever pressing ceases entirely, the response is said to have been ___.
Extinguished
If lever pressing has not yet ceased entirely, then the response has been only ___ ___.
Partially extinguished
What is the critical first step in extinguishing a behaviour? Give an example.
- Determining the effective reinforcer that is maintaining a behaviour
→ Child wines (R) -> Gets candy (Sr)
→ Child wines (R) -> Attention from parent (Sr)
→ it might not be necessarily getting candy which is reinforcing the child’s whining, but rather the accompanying attention from the parent
→ if it’s the latter, then withholding the candy might have little or no effect on the behaviour (in this case, the parents attention is the reinforcer, rather than withholding the candy)
→ another possibility is that whining is being maintained by both the candy AND attention, in which case withholding the candy would only partially extinguish the behaviour
→ this is why determining the effective reinforcer is a critical first step
→ Child wines (R) -> Gets no candy + Gets no attention (no reinforcer) -> no more response
When an extinction procedure is implemented, it is often accompanied by certain side effects that seem a lot like the 5 stages of grief; what are these side effects?
→ extinction burst (denial)
→ increase in variability (denial)
→ emotional behavior/frustration (anger)
→ aggression (anger)
→ resurgence (bargaining)
→ depression (depression)
→ acceptance ? (not a side effect of extinction)
What is extinction burst?
- A temporary increase in the frequency and intensity of responding when extinction is first implemented
→ Rat will press the bar rapidly when food is no longer delivered
→ The child may scream with increased enthusiasm when not rewarded.
→ Repeatedly hit the ’open door’ button when the elevator won’t move.
→ You push ‘on’ button repeatedly when your phone won’t respond
What is an increase in variability as a side effect of extinction?
- Trying a variety of methods for attaining the reward
→ Rat will try pressing the side of the bar, or using the left paw instead of the bar rapidly when food is no longer delivered
→ You rapidly push all the apps on your phone when it won’t respond
→ You hit all the buttons in the elevator when it won’t move
What is emotion behaviour/frustration as a side effect of extinction?
- We typically refer to these behaviors as frustration
- Examples:
→ Agitated behavior of a the rat
→ Screaming in the stuck elevator
→ Yelling at your phone
What is aggression as a side effect to extinction?
- An emotional response to extinction.
- Examples:
→ A rat whose lever pressing is placed on extinction may attack another rat
What is resurgence as a side effect of extinction?
- The reappearance during extinction of other behaviors that had once been effective in obtaining reinforcement
→ A rat may begin digging in the ground because it has previously found food in its bedding
→ You may try holding the power switch and the ‘on’ button
What is depression as a side effect to extinction?
- Low activity is a common symptom of depression.
- Example:
→ The hungry rat withdraws into the corner of the cage.
→ The guy with the broken iPhone withdraws into the corner of his cubicle - often associated with the loss of reinforcement, so if someone dies that is important to you and was a major source of reinforcement, you are essentially experiencing extinction and will become depressed over time
What are some problems with extinction side effects?
- Side effects of extinction impede successful implementation of extinction
- Side effects can be inadvertently strengthened if one suddenly gives in and provides the subject with the sought-after reinforcer
→ p.ex: if a parent suddenly gives in and buys the candy after their kid throws a temper tantrum, then the mother has essentially reinforced the wrong behaviour
→ parents can inadvertently strengthen temper tantrums - Kids who think it’s acceptable to throw temper tantrums become adults who think it’s acceptable to throw temper tantrums
What is resistance to extinction?
- The extent to which responding persists after an extinction procedure has been implemented
- A response that is very persistent is said to have high resistance to extinction
→ p.ex: Begging continues for over an hour = high resistance - A response that disappears quickly is said to have low resistance to extinction
→ p.ex: Begging stops after 5 mins = low resistance
Persistence of responding may be influenced by the presence of a ___ ___.
Discriminative stimulus
→ Kid (Sd): Dog begs for food (R) -> Begging will be persistent (high)
→ Mother (Sdp): Dog begs for food (R) -> Begging will extinguish (low)
In relation to resistance to extinction, when trying to modify behaviour, ___ in the ___ of behaviour is important.
- Consistency in the outcome
→ Kid (Sd): Dog begs for food (R) -> Gets table scraps (Sr)
→ Mother (Sdp): Dog begs for food (R) -> Gets yelled at (Sp)
→ p.ex: you don’t say no to a kid for a few minutes and then give in, you stay consistent
Resistance to extinction can be affected by a number of factors, what are they?
- Schedules of reinforcement
- History of reinforcement
- Magnitude of reinforcer
- Degree of deprivation
- Previous experience with extinction
- Distinctive signal for extinction
Explain how schedules of reinforcement can influence extinction.
- Scheduled reinforcement is the most important factor influencing resistance to extinction
→ this is indicated by the partial reinforcement effect - Partial reinforcement effect
→ behavior maintained on an intermittent schedule will extinguish more slowly than behavior maintained on a continuous schedule
→ Resistance to extinction is particularly strong following a variable ratio schedule
→ the randomness in the schedule itself is providing strong resistance to extinction
Explain how the history of reinforcement can influence extinction.
- The more reinforcers an individual has received for a behavior, the greater the resistance to extinction
→ p.ex: 100 reinforced trials vs. 3 reinforced trials - Easier to extinguish an unwanted behavior when it first becomes evident
→ Kid (Sd): Dog begs for food (R) -> Gets table scraps (Sr)
→ Dog begs for food (R) -> Begging will be persistent (high resistance)
→ p.ex: If timmy whines for candy for a long time (several weeks) and they’ve gone through 100 reinforcers, it’ll be harder for him to quit (extinguish), than Alex who just recently started watching him and imitating this behaviour
Explain how the magnitude of the reinforcer can influence extinction.
- Large-magnitude reinforcers result in greater resistance to extinction than small-magnitude reinforcers
→ p.ex: small bag of food may not orient dogs attention as quickly as a large bag of food
Explain how the degree of deprivation can influence extinction.
- The greater the level of deprivation, the greater the resistance to extinction
→ A rat that is only slightly hungry will cease lever pressing more quickly than a rat that is very hungry - The ‘motivating operation’ plays a role in the susceptibility to extinction
Explain how previous experience with extinction can influence future extinctions.
→ When sessions of extinctions are alternated with sessions of reinforcement, the greater the number of prior exposures to extinction, the quicker the behavior will extinguish during subsequent exposures
Explain how a distinctive signal for extinction can influence extinction.
- Extinction facilitated when there is a distinctive stimulus/discriminative stimulus (SD) that signals the onset of extinction
→ Mother (Sdp): Dog begs for food (R) -> Begging will extinguish (low resistance)
What is differential reinforcement of other behaviour (DRO)?
- Reinforcement of any behavior other than the target behavior that is being extinguished
- Extinction can be greatly facilitated by both extinguishing the target behavior and reinforcing the occurrence of a replacement behavior
What is differential reinforcement of incompatible behaviour (DRI)?
- Reinforcing a behavior that is specifically incompatible with the target behavior
- With DRO & DRI, there is no deprivation of reinforcement and thus the likelihood of side effects is greatly reduced
→ p.ex: paying attention to a child if they’re doing anything other than fighting with their sister is a DRO procedure, and paying attention when they’re acting really friendly with their sister is a DRI procedure
The presence of a discriminative stimulus reliably affects the ___ of the behaviour.
Probability
→ the DS does not automatically elicit the behaviour, it merely signals the availability of reinforcement, thereby increasing the probability of behaviour
- Example:
→ A 2,000-Hz tone signals that lever pressing will lead to food
→ 2000-Hz tone (Sd): Lever press (R) -> Food (Sr)
→ the rat will thus learn to press the lever when hearing the tone
Behaviors (responses) that are likely to occur is the presence of a Sd are under ___ ___.
Stimulus control
How can apply the use of stimulus control?
- Stimulus control can be used to eliminate problem behaviors
- Putting a behavior ‘on cue’ minimizes the likelihood of the behavior occurring at other inappropriate times (in the absence of the cue)
- Say you want to minimize the time you spend on Facebook.
→ A push notification (cue) tells you you have a message.
→ Only check your Facebook account when you get a notification - Stimulus control can be used to create effective study environments
→ where you study can allow you to focus more or less
→ it can be a stimulus control in the sense that every time you’re in that environment, it allows you to be more productive
→ Skinner (1987) recommends establishing a particular setting that is used only for studying.
→ Your desk becomes the cue for ‘studying’
What is stimulus generalization?
- The tendency for an operant response to be emitted in the presence of a stimulus that is similar to the original SD
- The more similar the stimulus, the stronger the response
→ the rat will still lever press to lower tones
What is the generalization gradient?
- The strength of responding in the presence of stimuli similar to the SD varies along a continuum
- A flat gradient indicates strong generalization (left)
→ indicates that responding drops gradually as the stimuli becomes increasingly different of the Sd - A steep gradient indicates weak generalization (right)
→ indicates that the rate of responding drops sharply as the stimuli becomes increasingly different from the Sd
What is stimulus discrimination?
→ The tendency for an operant response to be emitted more in the presence of one stimulus than another.
→ A flat gradient indicates strong generalization/weak discrimination.
→ A steep gradient indicates weak generalization/strong discrimination
What is discrimination training?
- Reinforcement of responding in the presence of one stimulus (the SD) and not another stimulus (S△)
- A discriminative stimulus for extinction (S△) is a stimulus that signals the absence of reinforcement
- Example:
→ Whenever the 2,000-Hz tone sounds, a lever press produces food; whenever the 1200-Hz tone sounds, a lever press does not produce food
Explain the peak shift effect.
- The peak of a generalization gradient following discrimination training will shift from the SD to a stimulus that is further removed from the SΔ
→ this is just an exception to the general principle that the strongest response occurs in the presence of the Sd - Example: first train rat to press lever in presence of 2000 htz tone → then conduct test for generalization across range of tones varying in pitch (1200-2800)
→ prior to discrimination training: figure to the left
→ in bottom right figure: then submit rat to discrimination lever procedure: reinforce lever pressing in presence of 2000 htz tone and not 1200 htz tone
→ then again test for generalization across tones (this time rat only responds in the presence of 2000 htz tone, and not 1200 htz
→ prior to discrimination training, gradient/slope relatively flat → but once 1200 htz tone established delta, the strongest response occurs in the presence of stimulus further removed from delta (as demonstrated by gradient to the right)
What are multiple schedules ?
- Two or more independent schedules presented in sequence, each resulting in reinforcement and each having a distinctive SD
- Example:
→ A pigeon is presented with a VR 20 schedule on a green key, followed by an FI 10-sec schedule on a red key
→ Green key (Sd): Peck (R) -> Food/Red key (Sr / Sd): Peck (R) -> Food (Sr)
→ p.ex: in order to do well in a chem exam, you’ll study to get an A
What is a chained schedule?
- A type of multiple schedule of behavioural contrast
- A sequence of two or more simple schedules, each of which has its own SD and the last of which results in a terminal reinforcer (delivered only at the end, contrary to the multiple schedule)
- Must be completed in a particular order
- Example:
→ A pigeon is presented with a VR 20 schedule on a green key, followed by an FI 10-sec schedule on a red key, which then leads to the terminal reinforcer of food
→ Green key (Sd): Peck (R) -> Red key (Sr / Sd): Peck (R) -> Food (Sr)
What is the difference between multiple schedules and chained schedules?
-
Multiple schedules:
→ 2 or more schedules completed in a sequence
→ each schedule has a reinforcer -
Chained schedule:
→ 2 or more schedules completed in a sequence
→ There is a terminal reinforcer
Define behavioural contrast.
- A change in the rate of reinforcement on one component of a multiple schedule produces an opposite change in the rate of response on another component
- As the rate of reinforcement on one component changes in one direction, the rate of response on the other component changes in the other direction
In a behavioural contrast, what is a negative contrast effect?
- An increase in the rate of reinforcement on one component produces a decrease in the rate of response on the other component
- Example:
→ A pigeon is on a multiple VR 60-sec VR 60-sec schedule and the VR 60-sec component on the red key is changed to VR 30-sec.
→ What will happen to the green key responding?
→ With more reinforcement now available on the red key, the pigeon will decrease its rate of response on the green key
In a behavioural contrast, what is a positive contrast effect?
- A decrease in the rate of reinforcement on one component produces an increase in the rate of response on the other component.
- Example:
→ A pigeon is on a multiple VR 60-sec VR 60-sec schedule and the VR 60-sec component on the red key is changed to VR 120-sec
→ What will happen to the green key responding?
→ With less reinforcement now available on the red key, the pigeon will increase its rate of response on the green key
In the contrast effect, as the one alternative suddenly becomes ___, the remaining alternative, which is still reinforcing, is viewed as relatively more ___.
Punishing; attractive
What is anticipatory contrast?
- The rate of response varies inversely with an upcoming (“anticipated”) change in the rate of reinforcement
- Examples:
→ Pigeons increased their rate of responding for reinforcement when presented with a discriminative stimulus for extinction
→ Things we are about to lose often seem to increase in value
→ p.ex: lindsay views her relationship with Bryce as dull, but she learns that he might be starting to like someone else, so she starts seeing the relationship in a better light
What is spontaneous recovery?
- Spontaneous recovery is the reappearance of an extinguished response following a rest period after extinction
Errorless discrimination training reduces many of the ___ ___ and ___ associated with discrimination training.
Sife effects; errors
