5- Reinforcement history: implications for a clinical treatment and experimental design Flashcards
The influence of Past Events on current behavior.
Behavior that persists in particular contexts during EXTINCTION
Responding that occurs at unnecessarily high or low rates to obtain reinforcement
Rule-governed bx That doesn’t match current contingencies
Reinforcement history
Your clients and research participants will have existing reinforcement histories.
These histories May influence responding during your assessments and interventions, resulting in:
Responding in ways that you did not predict
Unsuccessful treatment attempts
Less rapid changes in responding than predicted
Reinforcement history influences what we do. (We don’t start with a blank slate every day)
A central tenet of behavior analysis
Can be EXACERBATED in certain conditions.
Can pose threats to the INTERNAL validity of experiments.
Reinforcement history effects
- Can control the previous history with the response and reinforcer (through the use of naïve animals)
- Can control extra-experimental history during
the experiment (standard light-dark cycles,
controlled access to reinforcers outside of
sessions) - Don’t have to worry about influence of verbal
behavior
Why studies used NON-HUMAN animals
Reinforcement History Studies
Typically examined histories with different reinforcement schedules on CURRENT responding.
- Done by providing a history with two or more
“history SCHEDULES”.
-Responding on target schedule can then be assessed to determine extent to which history persists.
Reinforcement history studies
Historically, used as target schedules
May be particularly SENSITIVE to reinforcement history effects
-
Responses during the interval do not influence delivery of the reinforcer.
Response rates can vary widely without influencing reinforcement rate.
Therefore, they don’t select against particular rates or patterns of responding. If we establish those response rates, For instance, high or low or low rates in a history schedule or scalloped or break and run patterns we can look to see the extent to which these rates or patterns PERSIST during FI target schedules
Fixed Interval Schedules
Established History Effects on Reinforcement Schedules
NATURALLY Occurring schedules maintaining behavior may share features with FI schedules
FI schedules May be used for acquisition and Maintenance of appropriate behavior.
Why History Effects with FI Might
Be Important for Treatment
Influence two kinds of extinction
In different ways
Reinforcement history
…Highly sensitive to behavioral history.
Wiener study:
Exposure to FR or DRL schedules before FI Schedules dramatically affected FI performance.
(FR, produces very HIGH Rates of responding and DRL ProduceS very LOW response rate.)
Found Even REMOTE history seems to affect responding
History may be more influential when particular histories are CORRELATED with the distinct stimuli. Ie, room, color.
Study Suggests that CURRENT contingencies determine response rate in conjunction with previous reinforcement history
Weiner: Response rates were much higher on the FI schedule following the FR histories then following the DRL histories.
Responding during FI schedules…
Weiner’s study:
Should be considered when different treatment effects are observed across participants or across REPLICATIONS.
Could be used to improve intervention outcomes
Reinforcement history
manipulate rate on interval schedules by arranging for particular reinforcement histories
—Might generate higher rates of responding during that interval schedule as a function of the previous ratio schedule
Might be useful for academic improvement.
• Permit SHIFTS to interval schedules after establishing histories with DRA on RATIO schedules •••may create bias toward appropriate behavior
To produce those HIGH rates of responding During INTERVAL Schedule, Could;
Highly sensitive to behavioral history.
Responding during FI schedules.
Ono Iwabuchi study: non human: Rates of REMOTE history of FR, DRL schedules sustained when interval schedule introduced
IMPLICATION for Application: Residential TM
• Child’s Problem behavior reinforced on a DRH-like schedule in the home
Then, experiences treatment in a residential setting which results in decreased rate of problem behavior.
• Then Treatment implemented in home Following residential setting may result in HIGH response rates.
- likely to occur if a relatively WEAK- Schedule is used in treatment, even if treatment is implemented with high Integrity.
Ono Study
History effects during common treatment schedules (DRA, DRO, NCR)
Histories associated with particular stimulus
conditions in treatment contexts carries over to naturalistic Settings.
Extent to which stimulus conditions can be gradually shifted to promote or reduce history effects
Extent to which there are species differences in durability of behavioral history (more durable with human participants?)
Further research needed
Not obtained just with FI schedules.
•Alleman and Zeller:
Responding on FT Schedules following DRL or FR histories
.Initially Response rates were…
-Low during FT after DRL
-High during FT after FR
Remote history played a role
FR - DRL -FT
Response rates during FT were “ high”.
-FR more durable
Follow up study: FR then DRL then FT.
Response rates:
- high during FT
-Low during DRL
-High during FT - Suggest that remote history by the FR schedule continued to influence responding even though there was an intervening DRL schedule.
implications applications;History Effects
Refers to request that are likely to result in compliance from client.
The sequence involves Repeated presentation of high- P requests with a few interspersed low -P Request.
Reinforcement given for COMPLIANCE, typically on a FR1 schedule
- increases compliance with low – P requests as a function of events in the clients immediate history.
But effects are short lived
High Priority - High-P
May influence responding during your assessments and interventions
Client Histories
Client Histories may influence responding during your assessments and interventions resulting in:
Responding in ways that you did not PREDICT:
- Unsuccessful treatment attempts
- Less rapid CHANGES in responding than predicted
Can pose threats to the internal validity of experiments
Historical variables
Dependent on the previous reinforcement of the organism.
All extinction effects
- Respondent
2. Operant
Extinction- 2 types
Deals with reflective responses that are elicited by antecedent stimuli.
When a previously neutral stimulus is paired with an unconditioned stimulus, the previously neutral stimulus, Now: conditioned stimulus, will elicit a response similar to that elicited by the unconditioned stimulus.
Respondent (Classical conditioning)
No longer Pairing the stimuli, which results in the condition stimulus no longer producing the conditioned response.
Respondent extinction
Deals with the voluntary responses That are part of Contingencies.
Operant conditioning
Involves no longer providing the reinforcement dependent on the response.
This Results in decreased response rates by Breaking the contingency.
Operant extinction
Similar in that they both reduce the frequency of responding as a result of disrupting events that occurred contiguously In the environment.
Differ in the type of response that is disrupted and the type of disruption that occurs
Respondent extinction
Operant extinction
Here pairing does not occur Following any particular response.
It is Purely antecedent stimulus to the presentation of food, stimulus, and the body’s response to the presentation of food, response.
- S-R relationship. Purely respondent conditioning
However..
- Can we say praise becomes capable of functioning as a reinforcer is strictly through responded conditioning even though we say that the food item with which it paired has already been defined as a known a reinforcer? There is Important operate history here but more analysis is required
- A reinforcer does not function as a reinforcer strictly because it elicits a reflex such a salavation.
- Must be an EO that increases the value of the stimulus: typically for food, The EO is deprivation.
But even in the AO condition, when you have a full stomach you may still reflexively salivate to food on your tongue, but in this case the presentation of more food might actually function as a punisher.
And what is praise paired with the food here? The conditioned function of praise Would now depend on the related motivational Operation, NOT Strictly on it’s being paired with the elicited reflex response of salvation
Pairing to Develop a Conditioned Reinforcer
- Can we say praise becomes capable of functioning as a reinforcer is strictly through responded conditioning even though we say that the food item with which it paired has already been defined as a known a reinforcer? There is Important operate history here but more analysis is required
Pairing with a “Known” Reinforcer
- A reinforcer does not function as a reinforcer strictly because it elicits a reflex such a celebration.
- Must be an EO that increases the value of the stimulus: typically for food, The EO is deprivation.
But even in the AO condition, when you have a full stomach you may still reflexively salivate to food on your tongue, but in this case the presentation of more food might actually function as a punisher.
And what is praise repaired with the food here? The conditioned function of praise Would now depend on the related motivational Operation, NOT Strictly on it’s being paired with the elicited reflex response of salvation
Pairing to Develop a Conditioned Reinforcer
- This, we would remind ourselves that items are not reinforcers. Reinforcement is a process, defined by its affect, not a static thing that exists in the environment.
Stimuli sometimes functions as reinforcers following certain responses based on many…
Value altering motivational Variables.
And MO’s may be antecedents, but they are directly related to consequences. They are OPERANT variables, NOT RESPONDENTS!
Pairing to Develop a Conditioned Reinforcer
Increase in rate as a result of reinforcement
Operant Response
Respondent (classical conditioning) deals with reflexive responses that are elicited by antecedent stimuli
When a previously neutral stimulus is paired with an unconditioned stimulus, the previously neutral stimulus (now: conditioned stimulus) will elicit a response similar to that elicited by the unconditioned stimulus
Extinction:
•Involves no longer pairing the stimuli
•Results in the conditioned stimulus no longer
producing the conditioned response
Respondent Extinction
History Effects During Extinction
Operant conditioning deals with voluntary responses that are part of
contingencies
Operant responses increase in rate as a result of reinforcement
Extinction:
-Involves no longer providing the reinforcer
dependent on the response
-Results in the decreased response rates
(breaking the contingency)
Operant Extinction
Gretchen “praising” as she delivers food is pairing praise with the food,
and thereby eliciting salivation – and all of the unconditioned physiological sensations that accompany the presentation of food
Thus, Dr. St. Peter defines this as a strictly respondent conditioning process
Back to the ASR and Gretchen
Development of a Conditioned Reinforcer or Punisher (Sr or Sp):
In 2004, Jack Michael stated: “An ineffective stimulus is paired with a stimulus that already functions as a reinforcer or a punisher (either
unconditioned or conditioned). The procedure is the same as with respondent conditioning, …
but the desired outcome is a stimulus that will function as a reinforcer or a punisher rather than a
stimulus that will elicit a response similar to what the effective stimulus elicited.” (p.88)
Development of a Conditioned Reinforcer or Punisher (Sr or Sp):
Alessi,
Alternative way to condition a reinforcer
•Pre-school children received M&Ms for good work.
-worked to reinforce the on-task behavior of many of the children.
Then shown pieces of yellow paper cut into squares, and told that “this is what the big kids work for.”
The children worked for the yellow paper.
No pairing occurred here…
Verbal Analogue Conditioning research
Function as reinforcers across a WIDE range of Motivating Operation conditions.
Ex, Praise
The more reinforcers with which this has been paired, the greater the likelihood that it will be effective at some point in time
Generalized Conditioned Reinforcers
Breaking a response-reinforcer
dependency that results in GRADUAL reduction in response rate.
Operant Extinction
Similar to punishment in that it reduces the rate of responding.
Dissimilar in that it does not involve response dependent stimulus change
Extinction
Similar to DRO in that it reduces the rate of responding
Although a common component of DRO, is not the only component as DRO typically involves the delivery of reinforcers dependent on the absence of target responding
Comparing Extinction to DRO
Most Important to know the behavior function to be effective.
Without knowing the reinforcer Maintaining responding, cannot be effectively implemented
Underscores the importance of a functional behavior assessment in the development of behavior change plans
Using Operant extinction
Process:
• Through Behavioral mechanism
• Effective and Evident When theres reduction of response rate following a break in response-reinforcer relation
PROCESS of Extinction
Will look very different depending on the function of the behavior.
the procedure must be linked to the function.
Regardless of what the procedure looks like, it will involve no dependency between the response and the reinforcer
Extinction: procedurei
Should be using combination with reinforcement procedures to ensure that you were building a Replacement response
Forms the backbone of differential reinforcement procedure but can be difficult for caregivers to Consistently implement
Reinforcement contingency should always favor the prosocial response
Extinction
Not just about response reduction
Known to generate several different kinds of behavior.
Extinction
Include other treatment procedures May reduce the likelihood of “bursting”
Gradual thinning the schedule may reduce ___ later
Explain and Warn caregivers that “behavior gets worse before it gets better”
Applying BST techniques such as role-play and coaching to train parents to appropriately implement procedures that include extinction is useful in order to improve initial treatment Integrity
Reducing extinction bursts
Likely to be a respondent behavior that is elicited by the extinction situation
Probably not immediately sensitive to consequences
• plan on how not to reinforce it
Include procedures for how to react if it occurs
Be sure procedures are doable for that client
Extinction-Induced Aggression ; (Generative extinction affect)
A generative extinction effect
The emergence of responses not previously Observed when extinction procedures are implemented.
Responses that emerge during this, May be appropriate or inappropriate and can form basis for SHAPING New behavior -These responses Could include variations on targeted Topographies and form response class hierarchies
Response variation
Can occur when treating problem behavior
(A response class is… “A group of responses varying in topography, all of which have the same effect on the environment” )
Consists members of a response class that occur in a consistent sequence when a response fails to be effective. These sequences are built through reinforcement history with a common Functional reinforcer
Response-Class Hierarchies
Plan for response variation
Take the time to Interview caregivers
Ask them about possible response class hierarchies.
Make a plan to reinforce Desired response variation
Safely manage undesirable response variations.
Implement the procedures with a highly trained Therapist before having caregivers implement.
When dealing with problem behavior
The recovery of previously treated responding
Typically occurs when there is some Disruptor to the treatment. Could be:
- A return to a context in which problem behavior was previously REINFORCED,
- Issue with INCONSISTENT implementation
* The addition of reinforcers previously associated with the problem behavior
Treatment relapse
- Renewal
- Inconsistent implementation of the treatment (Treatment integrity failures)
- Resurgence
Treatment relapses
A form of treatment Relapse associated with changes in CONTEXT
A treated behavior Returns even though extinction is still in place
Relapse tends to be brief unless it Contacts a reinforcer
Renewal
The extent to which procedures are Implemented as described.
Two types:
- Omission error- Treatment component not applied when it should have been
- COMMISSION Error -Inappropriate application of a treatment component
Treatment integrity failures
A form of treatment relapse associated with increased EXPOSURE to EXTINCTION
Often In clinical practice, is-due to a more Intermittent implementation of DRA.
The Previously treated behavior often returns even though extinction is still in place for that response.
Relapse may persist for sometime. More research needed
Resurgence
Planning for treatment relapse when..
- Reduce reinforcement ratio in a
Controlled context with a therapist - create a plan for possible treatment relapse with Implementation agents.
- Monitor integrity frequently and provide ongoing Feedback to promote high levels of integrity
- Include a more easily implemented Treatment component to reduce exposure to extinction
INTEGRITY may be a problem
Ensure there is no reinforcement of problem behavior during a relapse
If treatment relapse is a possibility
Revisiting Gretchen
At the beginning of this unit, we talked about Gretchen, a therapist who paired praise and edibles to condition praise.
She then used praise (without edibles) in
her teaching sessions.
What was your best guess?
We might see…
- Reduction in correct responding
- Potential extinction burst and emotional responding
- Response variability
- Response relapse
Most well known affective history during extinction
Response reduction during extinction happens more quickly after Continuous reinforcement then intermittent (partial) reinforcement
Can be examined several ways by looking at:
- Number of responses required to meet
predetermined extinction criterion
- Number of experimental sessions required to
meet criterion, OR,
- Proportion of responding during EXT To how much during baseline.
Partial reinforcement extinction affect
PREE
The influence of recent reinforcement history And it was causes behavior to change gradually during exposure to new contingencies. Decrease in historical influences across time can lead to attainment of steady state responding.What leads us to study state is that we are overriding that previous reinforcement history with the current contingencies. And when we reach steady state it’s because hopefully we have overwritten that previous history and we now have responding that is more completely under control of the current contingencies.
Transition states
Lerman et al. (1996):Examined PREE
They started with continuous or intermittent baselines
Examined Absolute response rates and proportion of baseline measures During extinction phases following each of the continuous or intermittent phases to determine whether extinction occurred more quickly following continuous or intermittent reinforcement.
Found evidence of REVERSE PREE for two participants (some evidence for third participant)
Maybe an artifact of different baseline response rates.
PREE
Maybe an artifact of different baseline response rates.
When applying, consider briefly reinforcing problem behavior on an FR1 before extension as might be done during a functional analysis anyway.
-May reduce overall response rates and result in more rapid Extinction of responding – but maybe not
PREE
Has been showing to be highly variable across studies.
Some studies found adorable history is particularly of DRL schedules: Weiner,
Other Studies found a greater influence by the most recent history
Might be due to differences in:
• experimental procedure , or ..
• extra experimental histories
Durability of reinforcement history might be highly influenced by strength of the target schedule (FI FT, EXT Most susceptible to history effects?)
Durability of history effects
history effects may define transition states.
Plan on implementing sufficient numbers of sessions to reduce influence
Directly access reinforcement history by building replication sets of conditions
Planning for REMOTE history effects in experimental design
Can result in sequential confounding: ( when One phase Follows another so effects Cannot be separated from history with a previous phase..
Extreme:
- sequential confounding characterized as irreversibility, (failure to withdraw or override history From a previous condition). - Most frequently a problem when independent variable results in skill acquisition, which cannot easily be withdrawn or overridden.
Even at Lesser degree, recent history can dramatically impact performance during current reinforcer contingencies.
Reversal designs
Systematically evaluate history using pairs of conditions.
However, this takes a long time to do. Consider trying to counterbalance the order of conditions across presentations, participants, or both.
Look for dwindling affects of an intervention across time.
When worried while planning for history in REVERSAL DESIGNS
Can be influenced by reinforcement history essentially through generalization across responses or settings
Desirable clinically, even though can Be bad for experimental control
Multiple baseline designs
Select baselines carefully
Choose related baselines but not TOO related
When GENERALIZATION may be a problem, Use a combination of designs such as;
- reversals
- multi element,
Consider using different kinds of multiple baselines.
Eg, across participants, across responses
Multiple baselines – planning for history Effects
less prone to extra experimental history, particularly if multiple conditions are conducted on same day/appointment
More prone to carryover/Alternation effects
- Carryover is likely when conditions are not Highly Discriminable from each other
May result in contrast Effects
Multi element designs
When there is a change in response rate in one component when changes are made to another component. Example: a home versus a school environment.
Not a ton of evidence
Contrast Effects
Use clear discriminable stimulus
Counterbalance the order of conditions; allows for assessment of potential carryover.
- look out for patterns following different condition orders.
Provide some time between sessions.
Frequent alternation may be more likely to lead to carryover While spaced sessions may reduce carryover effects (Powell)
multi element designs – planning for History effects
used when dramatic changes in response requirements would be contraindicated
Ie, Exercise, smoking, caffeine consumption
Changes in behavior are Dependent in part on past history of organism
May help to avoid ratio strain
Changing-criterion designs
Organism stops responding when reinforcement schedule is increased dramatically and abruptly
Ratio strain
Increase response requirements gradually
Establish response history at Intermediary steps
Do not stay too long. (A shaping process)
Changing- criterion designs: planning for history effects
Treatment component not applied when it should have been
Frequent errors are similar to applying to much extinction and that may result in Treatment Relapse
- Omission error:
Inappropriate application of a treatment component
- COMMISSION Error:
History may be more influential when particular histories are…
CORRELATED with the distinct stimuli. Ie, room, color.
may Be easier to implement during intervention then ratio schedules.
INTERVAL schedules
Weiner Study :
Determines response rate in conjunction with previous reinforcement history
Suggests that CURRENT contingencies
Weiner: Response rates were much higher on the FI schedule following the FR histories then following the…
DRL histories.
Not obtained just with FI schedules.
•Alleman,
Responding on FT Schedules following DRL or FR histories
.Initially Response rates were…
-Low during FT after DRL
-High during FT after FR
Remote history played a role
FR - DRL -FT
- Response rates during FT were “ high”.
- FR more durable
Follow up study: FR then DRL then FT.
Response rates:
- high during FT
-Low during DRL
-High during FT
Suggests REMOTE history by the FR schedule continued to influence responding even though there was an intervening DRL schedule.
implications applications;
-There may be some conditions under which the time based schedules, NCR, Are not effective due to reinforcement history.
Intermittent FR histories Could interfere with suppressive effects of time based schedules.
- •Baseline contingencies may be arranged to ENHANCE the efficacy of time based schedules.
- Clinicians should make reinforcement rate DIFFERENT between response dependent baseline and a time based schedules if seeking response SUPPRESSION.
Reinforcement history
Alleman Study:
Could interfere with suppressive effects of time based schedules.
Intermittent FR histories
Type of Response:
We are DISRUPTING the relation Within an ELICITED Response.
Type of disruption:
Disruption is in the Antecedent relation. So it’s between the unconditioned and conditioned stimulus.
- • Respondent Extinction
Type Response disrupted:
Disrupting a voluntary or evoked response
Type of Disruption that occurs.:
Consequent relation between the response and reinforcer.
Operant Extinction,
These response generative effects are typically considered…
Side effects.
may define transition states.
history effects
One phase Follows another. Cannot be separated from history with the previous phase.
sequential confounding:
- Aggression
- Emotional outbursts
- Response variation
- Treatment relapse a.k.a. as extinction burst
These response generative effects ( Side effects) of Extinction
May Be easier to implement during intervention then ratio schedules.
INTERVAL schedules
Could manipulate rate on interval schedules by arranging for particular reinforcement histories
Could Permit SHIFTS to interval schedules after establishing histories with DRA on RATIO schedules:
-May create BIAS toward appropriate behavior
Might have some implications for residential treatment.
Ex. Child Problem behavior is reinforced on a DRH schedule in the home. Child experiences treatment in residential setting which results in decreased rate of problem behavior. The treatment implemented in home following the residential setting may result in high response rate is. Likely to occur if a week schedule used in treatment even if treatment is implemented with high integrity.
To produce Ratio-like or HIGH rates responding (During INTERVAL Schedule),
There may be some conditions under which the time based schedules, NCR, Are not effective due to …reinforcement history.
-
reinforcement history
could interfere with suppressive effects of time based schedules.- -•
Intermittent FR histories
May be arranged to ENHANCE the efficacy of time based schedules.
-
Baseline contingencies
Clinicians should make reinforcement rate DIFFERENT between response dependent baseline and a time based schedules if..
seeking response SUPPRESSION.
increase in rate as a result of reinforcement.
Operant responses
A Take away point:
Strongly related to respondent
conditioning and..
May always have some connection to pure unlearned S-R relationships (reflexes)
However, the picture seems more complex, and when pairing occurs during operant conditioning
procedures, discussing pairing as a strictly respondent phenomenon may be an oversimplification
Pairing
similar to applying too much extinction and that may result in treatment relapse
Frequent omission errors are
Each design is more prone to different kinds of history effects. There are still relatively few studies directly examining effects of reinforcement history or likelihood of his three fax. There is more research needed on creating histories, evaluating the influence of those histories on different kinds of designs, Determining utility of combined designs to address we had for us in history facts, and means of reducing the impact of being
THE END!!!
