3- Reinforcement arrangements and applied to settings 1: selecting reinforcers and contingencies Flashcards
Simplest
arrangement; deliver a discrete consequence for each discrete response
Not common
Continuous reinforcement:
: Reinforcement only sometimes follows the target response
-more common
Intermittent reinforcement
Reinforcement is provided for the first response that follows some amount of time
Two types:
Fixed interval
Variable interval
Interval schedules:
Reinforcer delivered contingent on first response after some constant time period has elapsed
Fixed-interval (FI):
Similarly arrange reinforcers contingent on the first response after a specified time, but the time varies from one reinforcer to the next
Variable-interval (VI):
Arranges reinforcers for the last of a number of responses that varies from one reinforcer to the next.
Variable ratio
Reinforcement is provided following a pre-specified number of responses
There are two varieties of ratio schedules:
- Fixed Ratio FR
_Variable Ratio VR
Ratio schedules
Deliver reinforcer after the last of a fixed number of responses
Fixed ratio (FR
Deliver stimuli known in other contexts to be reinforcers solely on the basis of time, independent of responding
In a Fixed time schedule that Reinforcer is delivered after some constant time period has elapsed
A Variable time (VT) Similarly arranges reinforcers after a specified time, but the time varies from one reinforcer to the next
Response-independent time-based) schedule
Of low rate (DRL)t s without response, then 1 response
Of High Rate (DRH) 1 response within t s or less of last response
Of other behavior (DRO) t s without a response
Of alternative behavior (DRA) Contingent on alternative response, often in absence of target response
Some other basic Schedules
When a reinforcer follows a response, it’s effect depends on its relation not only to that response but also to other responses preceding it
All are followed by the reinforcer even
though only the last response produced it
E.g., if an error is immediately followed by
a reinforced correct response in an
instructional procedure, that reinforcer will
probably strengthen the error along with
the correct response, at least temporarily
Delay Contingncies
Response classes one wishes to
strengthen may vary in terms of how
well established they are.
From Not at all established to fully established but unmotivated
Selecting responses
Task materials and relevant instructions do not occasion predetermined level of performance in absence of prompts
Skill deficit
: The skill is intact, but the person is unmotivated to respond under appropriate stimulus conditions
can be accurately distinguished from skill deficits on basis of whether supplemental reinforcement for correct responding rapidly increases accuracy
Performance (or motivational deficit)
If reinforcement contingency very rapidly eliminates a performance deficit, the skill must already have been learned
Skills deficits, on the other hand, may
require different interventions depending
on the learner’s abilities and whether some of the skill has already been learned
Performance or Skill Deficit?
Some property of responding is gradually changed by differentially reinforcing successive approximations to target behavior
Extinction increases RESPONSE Variability (Some behavior must be already occurring )
Frequent reinforcer delivery can result in satiation and may strengthen early responses
Infrequent delivery may decrease or extinguish responding
Shaping
May aide in Shaping
Used as a systematic way to decide which approximations to reinforce and how often
Specifies changes in reinforcement CRITERIA (e.g., based on latency, effort, location or duration) as behavior moves toward the shaping target
To be reinforced, the next response must fall
into some PORTION of the sampled distribution
(e.g., above the 70th percentile)
The reinforcement criterion changes as progress is made toward the target
Lag Schedule is an example
Percentile Schedules
Use;
Extinction induced variation
Lag- reinforcement schedules
- A Type of percentile schedule
Lag = number of responses separating the current response from an earlier one like it
In a lag x reinforcement schedule, the
current response is reinforced if it differs
from the last x preceding responses along
the specified dimension from an earlier one like it
Ex, Shows an increased Variety of block building structures
Some additional training sometimes required.
To increase response diversity
Often used when a skill DEFICIT present
Multiple types of Prompting strategies least to most Verbal Gestural Physical guidance May include allowing some time for independent responding.
When prompting us required, question where in the sequence to deliver reinforcer
• Reinforcing only unprompted correct Responses may lead to INFREQUENT…., Reinforcers
•Reinforcing physically guided responses: Risk strengthening behavior that will not generalize beyond the learning setting
Prompting
Research has examined differential reinforcement of prompted and unprompted responding to achieve this goal, using:
- Different reinforcement schedules
- Different quality reinforcers
To Promote INDEPENDENT responding:
Ultimate goal of prompting is too…
Video ex;
Three conditions:
CRF/CRF (Independent and prompted, respectively
CRF/FR 3
CRF/EXT
Differential reinforcement: an unprompted response got food and praise while a prompted response got praise only.
Non differential reinforcement: Unprompted and prompted responses both get reinforcers and praise
Least to most prompting (example)
May be more RAPIDLY acquired or increase more reliably when Unprompted responses differentially reinforced (no reinforcement for
prompted responses). OR..
…Higher reinforcement rates arranged for
independent than for prompted responses
Independent responding
naturally related to the responses that produces
An intrinsic reward
Arbitrarily related to the responses that produce it. Example, music is an intrinsic outcome of playing an instrument, the music teachers praise Is extrinsic To the playing.
Extrinsic rewards
Reinforcement contingencies (Extrinsic rewards) Lead to decrements enjoyment (Intrinsic rewards) and thus result in decreases in engagement.
Fueled controversy regarding use of reinforcement in educational settings
..”Extrinsic motivators – including A’ S, sometimes praise, and other words are not merely an affective over the long haul but counterproductive with respect to things that concern us most: desire to learn, commitment to good values, and so on
Alfie KOHN, educational leadership
Strongest detrimental effect when tangible given for performance independent delivery
Overjustification hypothesis-
Extrinsic reinforcement and intrinsic reinforcement
Meta-Analysis of available research using effect sizes.
Separated effects according to reward type:
Reward for what (Quality dependent, completion dependent, performance independent What sort of reward (Tangible, verbal)
Examined separate affects on ENGAGEMENT (free time) And attitudes towards task
Calculation of effect sizes:
Experimental group mean - control group mean divided by pooled standard deviation
Overjustification study:Eisenberger & Cameron (1996):
Interpretation of effect sizes:
EF < 1 EFFECT of the independent variable: (detrimental effects Of reward)
EF > than 1 = no effect of the independent variable (No detrimental effects)
Eisenberger & Cameron (1996): Overjustification
SATIATION especially when reinforcers increase task engagement or less and motivation is examined for brief periods immediately after reward period.
(Overjustification ) Effects In IDD
What may account for what appears to be less and intrinsic motivation for people with developmental disabilities?
Examined role of concurrently available ALTERNATIVE activities and SATIATION in “Overjustification
General procedures: (A) Baseline: No programmed consequences for engagement with target activity
(B) Reinforcement: Piece of preferred food
contingent on 30 s engagement with target activity
(A) Return to baseline: No programmed
consequences for engagement with target activity
Overjustification
Peters & Vollmer (2014):
- Contrast Effects
2. Learned helplessness
Accounts for intrinsic motivation And learned helplessness phenomenon
- Changes in rate of reinforcement under one condition that can produce an opposite change in rate of responding in another condition.
Contrast affect
In performance-independent reward, “It
doesn’t matter how well I do
Learned helplessness phenomena
Over justification effect
More pronounced when behavior already occurs at a high level
• We generally do not arrange reinforcement contingencies for behaviors already
occurring at high rates
Reinforcement systems depend on task completion, performance quality, or both…reward procedures not reliably found to reduce intrinsic task interest
Some effects may be best attributed to satiation, especially when reward does increase engagement in the response and the effects are measured immediately afterwards
Quality-dependent verbal rewards actually have a positive effect on intrinsic interest
Addressing Overjustification hypothesis
Just a prediction
Prediction Conducted under extremely LOW effort requirements, often with simplistic responses
Predictions made under these conditions do not necessarily guarantee the utility of the stimulus in
actual training or treatment
Therefore, the predictions of These should be verified under conditions that more closely parallel actual training contexts such as:
Actual work , Realistic schedules
…Enter, reinforcer assessments…
Stimulus preference assessments
SPAs
General experimental arrangements:
Single-operant arrangement
Concurrent-schedule arrangement
Progressive ratio schedule arrangements
Demand curves
reinforcer assessments
most commonly used PA
Single reinforcement schedule is Arranged for a single response
FR schedule
DV = response rate or frequency
Compare to baseline rates
Single-operant arrangement
In theory, in this PA, the larger the increase in responding, the more potent the reinforcer
However, response rate may be poor index of relative reinforcer value because variables other than the strength of the reinforcer can impact responding
E.g., “ceiling effects”
mask differences in relative reinforcer efficacy
Single-operant arrangement
More sensitive test of relative reinforcer
effectiveness
Multiple response options CONCURRENTLY
available, Each associated with a distinct stimulus
delivered contingent upon appropriate response
DV is relative response allocations across
available alternatives
Often a control condition
Concurrent-Schedule Arrangement
Relative versus absolute value:
•may be sensitive to small differences in reinforcer value
•However, just because a reinforcer is less
preferred in This arrangement, it may nonetheless be an effective reinforcer in an absolute sense
• May mask reinforcer effects for lower preference stimuli
Concurrent-Schedule Arrangement
Special type of single-operant arrangement
• Differ from typical single-operant arrangements with respect to how the schedule is thinned
-Ratio requirement increases systematically
within a session, across successive “trials”
-Session ends when participant ceases to respond for pre-determined amount of time
Dependent measure, aka, break point,” is the value of the last completed schedule requirement
Provide an estimate of the amount of responding one is willing to emit towards gaining a reinforcer
Stimuli of different preference levels may produce equal results in how fast an individual might work But different results and how much work is done.
Progressive Ratio Schedule Arrangement
When all else being equal As unit price increases Demand (consumption) decreases And vice versa
Demand Curve:
Relates to Unit price of the commodity and.,the Total amount of a reinforcer that is consumed
Allow one to look at choices under conditions of asymmetricalEnforcers and under various conditions of constraint.
Law of Demand:
Provides an estimate of the amount of responding one is willing to emit towards gaining a reinforcer
Stimuli of different preference levels may
produce equal results in how “fast” an individual might work, but different results in “how much” work is done
break point
methods used to identify
stimuli that may function as reinforcers
Prediction:
The Higher the preference of stimuli the more
effective the reinforcers
Two step process:
1. Conduct SPA to determine its predictions
about relative reinforcer efficacy
- Test those predictions using one of
the REINFORCER assessment methods just
described
There are three general categories.. see next card
Stimulus Preference Assessments
Three general categories
- Indirect preference assessment (client or caregiver report)
- Naturalistic (in-vivo) DIRECT observation
- Direct preference assessment (free
operant or approach based)These differ in effort and accuracy
Stimulus Preference Assessments
Indirect PA
Advantage- Least effortful
Limitations: subjective, results do Not always correspond to direct PA’s
Naturalistic observation
Advantage: objective, more valid than indirect PA
Limitations: time-consuming, limited to stimuli available and natural environment
Direct PA
Advantage: most accurate, stimuli not limited
Limitations: time-consuming
Preference assessment – effort and accuracy
Compare a teacher and stimulus preference assessment ranking
Found a strong positive correlation of 1/9 cases and a negative correlation of 5/9 cases.
Presented HP stimuli contingent upon
occurrence of a response
Conclude: Direct assessment methods
more accurate in determining reinforcers than
Indirect assessment
Accuracy of Caregiver Rankings
Cote, Thompson
During this preference assessment, items are presented systematically to produce preference HIERARCHIES
Variations differ with respect to how many items are presented during a given trial
Preference hierarchies are derived from calculations of the number of times a stimulus is SELECTED given the number of times each is available
Approach or selection-based preference assessments
Identifying Stimulus Preferences
Place items, one at a time in front of the person
• Typically, 10 trials per item
Measure whether or not they approach the item
Hierarchy based on: Approaches/trials
Simple, includes large number of stimuli but may be prone to false positives.
Single-Item/Approach Method/AKA single-stimulus assessment
Stimuli presented in pairs
Continue to present item in pairs until Each item has been paired once with every other item (all pair-wise permutations)
Number of trials = n (n-1) / 2,
where n = total number of stimuli included
E.g., for 5 stimuli, 5 (4) / 2 = 10 trials
Record which of the two items they approach including:!
• Selection = brief access
• Attempts to select both that were blocked
Hierarchy based on:• Approaches/trials
More sensitive To relative preference and also includes a large number of stimuli but is time consuming And there is a potential for a side bias
Paired-stimulus PA (AKA – forced-choice PA)
Present all items in the array
Simultaneously
Participants select one from among all items (or remaining items) during each trial
Two Variations.
1. With replacement (MSW) –
• Selected items are returned to the array
-Provides info on the single most preferred stimulus
- Without replacement (MSWO) –
• Selected items are NOT returned to the array
-Provides more information about preferences among the array
Brief but may limit the number of items included and it can be prone to save the best for last.
Multiple-Stimulus PAs
Reinforcing PHYSICALLY guided responses risks strengthening behavior that will not….
generalize beyond the Learning setting.
Used when target responses are
sufficiently complex or of sufficiently low probability that they are unlikely to be emitted without this gradual intervention
Shaping
Type of Percentile Schedule
- Lag = number of responses separating the current response from an earlier one like it
In a lag x reinforcement schedule, the
current response is reinforced if it DIFFERS
from the last x preceding responses along
the specified dimension
Ex., Lag 3
Lag Reinforcement Schedule
Items are presented to the individual and the proportion of time spent engaging with the items is recorded
Arranged in two ways:
- For Free operant (or multiple item) Cesc meant all items are presented SIMULTANEOUSLY
- In Single item presentation each item is presented INDIVIDUALLY Several times.
Duration based preference assessment
Variation of duration based assessment
Used to determine the extent to which stimuli displace problem behavior.
Two measures are taken simultaneously:
- Stimulus engagement
- Problem behavior
Stimuli selected based on combined it measures
Competing stimulus preference assessment.
Use vocal or pictorial presentations:
Vocal assessments: ask what they want
Pictorial assessments – when you present pictures of stimuli
Use for Inclusion of the complex stimuli not easily presented on table top such as community activities, bike rides or swimming
Variables to consider:
Abilities
Position bias
Problem behavior
Require different skills to make Valid selections. important to consider Pre-requisite skills specific to each type of assessment In relation to participants current skills with selecting a method
Selecting a preference assessment
Single stimulus - Approach response
Paired stimulus – approach response, visual scanning, possible position bias
Multiple stimulus – approach response, visual scanning, possible position bias
Duration – approach response
Pictorial SPA - Auditory match to sample
Prerequisite skills to consider when selecting a preference assessment method
Individuals with profound disabilities who do not possess prerequisite scanning and motor skills for approach response cannot participate in traditional SPAs
Other options
- Microswitches ) - Indices of happiness
SPA
Trained individuals to email small motor movements e.g.. Lifting head, to assess stimuli
Attached to body parts then measured number and duration of motor movements.
Microswitch
Identified high preference and low preference items in SS SPA
presented for 1 to 3 minutes, 2 times per session and measured indices of Happiness and unhappiness
INDICES of happiness and unhappiness
Selection is controlled by location rather than by the items themselves
Participants I always select the item on the left went to items are presented in the paired stimulus S PA.
Eliminating can be difficult but has proven successful in some cases
Position Bias
Bourret, Iwata Look up position biases for three individuals
- QUALITY training- Gave participants a choice between known non-preferred stimulus and preferred stimulus used in original SPA.
- Worked for us some not all.
• MAGNITUDE Training with error correction. ——Magnitude of one option was five times greater than the other
Error correction: selection of smaller magnitude resulted in five re-presentations of the trial, selection of small option was blocked
Eliminating position biases
Changing to VERTICAL placement of stimuli, holding stimuli directly in front of participant and placing items in opposite corners of Room and having participant walk to selected item.
If position Biases persist, it may be necessary to assess preferences using the SS or DURATION base procedures
Other possible methods to eliminate position bias
Can be distinguished from skill deficits on basis of weather supplemental reinforcement for correct responding rapidly increase accuracy
Performance deficit
Requires experience
-Some behavior is already occurring
- Frequent reinforcer delivery can result in satiation and may strengthen early responses
- Infrequent delivery may decrease or extinguish responding
Shaping
Reinforcer assessment in which the reinforcer. efficacy of a single item is tested by comparing levels. of a response when it is delivered as a consequence. to levels observed during baseline (no consequence)
Single Stimulus Preference Assessments, also known as “successive choice” assessments, are conducted by providing a single item to a child, and recording his behavioral response to each item, as well as the duration of his engagement with each item. Although Single Stimulus Preference Assessments may not be as accurate at determining preferences as MSWOs, MSWs, and Paired Stimulus Preference Assessments, these are appropriate for children who are unable to select between highly-preferred and low-preferred items. For example, if you conduct a Paired Stimulus Preference Assessment and noticed that the child always selects items from one side (i.e., side bias) or always attempts to take both presented items, a Single Stimulus Preference Assessment should be used instead.
Single-operant arrangement
Identified preferences w/ paired-stimulus assessment
Compared “accuracy” (how often known
preferred food chosen in 2-choice trials)
under 3 conditions
1. Object: Presented actual items
2. Spoken: “Do you want X or Y”
3. Picture: Presented pictures of the items
Examined correspondence of accuracy as a
function of abilities on the Assessment of
Basic Learning Abilities (AB
Conclude: Verbal and pictorial SPAs can be
accurate, but reserve them for individuals
with established discrimination abilities
Conyers et al. (2002):
Time
Function of problem behavior
Can stimulus be delivered after
selection
Other considerations when selecting
an SPA method
f, FO, SS, or MSWO may be more appropriate than PS assessment MSWO proposed as assessment that required less time to implement than a PS assessment • DeLeon and Iwata (1996): PS and MSWO methods generated similar preference hierarchies, but MSWO assessment required fewer trials and completed in approximately half the time
But, fewer stimuli can be assessed with MSWO, therefore: SS if one wishes to include a large number of stimuli PS if one has ample time to complete assessment
time is of issue
may impact
choice of SPA method
Kang et al. (2011): Analysis of interaction
between problem behavior maintained by
different reinforcers and different types of
SPAs
Children with problem behavior sensitive to
tangible, attention, and escape
Conducted 3 forms of preference
assessments (PS, MSWO, Free Operant)
Examine which forms interact with which
functions
Conclusion PS and MSWO evoke problem behavior maintained by tangible reinforcers • FO does not FO evoked problem behavior maintained by attention • PS and MSWO do no
Function of problem behavior
Selection responses typically result in the opportunity for a participant to
consume the chosen item
Under some circumstances, it may not
be practical to deliver an item
immediately following a selection
response (e.g., community activity)
Can this impact accuracy?
Hanley, Iwata, & Lindberg (1999):
Evaluated preferences using pictures
During each assessment trial, three pictures
were presented simultaneously to participants
Two potential reinforcers and presumably neutral
activity
Two experimental conditions:
No access: Selection did not produce
programmed consequences
Access: Selection resulted in 2 min access
In most cases, differentiated preference hierarchies
established only when selected items were
immediately delivered following selection
Considerations: Contingent
Delivery
Kuhn, DeLeon, Terlonge, & Goysovich
(2006):
Evaluated preferences using verbal SPA
Two experimental conditions
1. Verbal: “Would you rather have X or Y?”;
selection did not produce programmed
consequences
2. Verbal-plus-Tangible: Experimenter
presented two stimuli and asked, “Would
you rather have X or Y?”; selection resulted
in 30 s access
Followed by concurrent-schedule reinforcer
assessment
Different preference hierarchies generated
for all participants
During reinforcer assessments: Items ranked
high in the verbal-plus-tangible assessment
functioned as more effective reinforcers than
items ranked as highly preferred on verbal-
only assessments
Providing item does matter
Contingent delivery identifies stimuli more
likely to function as reinforcers
Contingent Delivery
What type of stimuli do you include in a preference assessment?
How do you pick which exact stimuli to
include?
Does it matter if you mix up all kinds of
items in the same assessment?
Can you use praise with children with
autism spectrum disorder (ASD)? How
do you even assess social stimuli?
Considerations: Reinforcer
Selection
Is it easily replenished?
Does it cost much?
Does it fit naturally in the environment in
which it will be used?
Can its use cause other sorts of detrimental
effects?
Does its effectiveness wane easily across
short periods of time?
Does its delivery disrupt ongoing behavior?
Ecological Fit of Reinforcers
Although indirect SPAs may not be all that accurate when used alone, some
advocate for their use in helping to
construct a stimulus array for direct
SPAs
Fisher et al. (1996):
Caregivers ranked standard list of items
Caregivers then completed RAISD and
rank ordered items from the RAISD
Paired-stimulus SPA then conducted
Results compared across 3 assessments
Fisher et al. (1996), results:
Top-ranked items identified by caregiver
predictions based upon RAISD more
preferred than the top-ranked items
identified by predictions based upon a
standard list of items
Thus, while caregiver reports may not
consistently identify the most preferred
items, they may play an important role in
constructing a stimulus pool that includes
the most effective reinforcers
Selecting Stimuli for Inclusion
What happens if you include different types of stimuli in the same SPA? DeLeon, Iwata, & Roscoe (1997): Conducted preference assessments with mixed (food and leisure items) arrays Repeated preference assessments, minus the food items Assessed whether initially LP activities functioned as reinforcers
Conclude Food items often downward displace leisure items in mixed arrays But those leisure items might be effective reinforcers nonetheless — assess separately!
SPA: Mixed Arrays
Arguments in favor of praise/social reinforcers Natural in the classroom Does not interrupt responding No cost other than caregiver effort Takes little time May be less subject to satiation Behaviors developed using social reinforcers may be more easily maintained in generalization settings May increase task interest
However, Praise not effective for all • Hence, it is important to assess as we do with other reinforcers Social reinforcers may be difficult in incorporate into preference assessment • May require use of pictorial SPA Children with ASD less sensitive to social stimuli as reinforcers?
SPA: Praise and
Other Social Stimuli
Preference for social stimuli among
persons with ASD
Somewhat mixed results
Celani (2002): Preference for contexts
devoid of social interaction
Call, Shillingsburg, Bowen, Reavis, &
Findley (2013): Children with ASD may
exhibit indifference towards various types
of social interactions
Goldberg, Allman, Hagopian, Triggs,
Frank-Crawford, Mostofsky, Denkcla, &
DeLeon (in preparation)
Children with and without ASD
Identified two groups of stimuli
• Social stimuli: Activities completed with mom
• Nonsocial stimuli: Activities completed by self
Then conducted pictorial SPA and PR
assessments with social and nonsocial
stimuli
Photos taken of participant engaging in activity
with mom (social stimuli only) or activity alone
(nonsocial stimuli only
Dangerous to assume that children with ASD will not find social interactions reinforcing As in all cases of determining reinforcer effectiveness, there are great differences across individuals
Social Reinforcers and ASD
Social reinforcers often difficult to include in preference assessments Smaby, McDonald, Ahearn, & Dube (2007): Conducted brief reinforcer assessments for social stimuli Rapid alternation Social consequence vs. extinction Eliminates difficulty of incorporating social stimuli in preference assessment
Assessing Social Stimuli
Dozier, Iwata, Thomason-Sassi, Worsdell, Wilson (2012):
Compared two pairing procedures
• Noncontingent pairing: Primary reinforcers
delivered freely and attention is consistently
provided during consumption
• Contingent pairing: Primary reinforcers and
attention simultaneously provided
contingent on the completion of a task
Following either pairing procedure, the
effectiveness of attention in the absence
of food is measured through task
completion
Noncontingent pairing ineffective
Contingent pairing more effective, but only in
about half of cases
Establishing Social Stimuli as
Reinforcers
- If the individual has the requisite visual
scanning and motor skills, PS
assessment should be used if time
permits, MSWO if time is limited. - If motor skills are intact but the individual
cannot visually scan an array of stimuli,
the SS assessment may be most
appropriate. - For individuals with side biases, the SS or
FO assessments should be considered if
training to overcome the side bias proves
ineffective. - For persons who lack the ability to visually
scan an array and who have limited motor
control, the use of technology (e.g.,
microswitches) or indices of happiness
could prove useful in aiding in the
identification of preferred stimuli. - Complex stimuli can be assessed through
the use of pictorial or verbal preference
assessments if the individual has the
requisite identity matching skills (visual
MTS for the pictorial assessment and
auditory MTS for the verbal assessment). - If the individual engages in problem
behavior maintained by access to preferred
stimuli, the FO assessment should be
considered. If the problem behavior is
maintained by attention, one should use
the PS, SS, or MSWO assessment. - When possible, always try to provide
access to the selected stimulus - When identifying the items to be included in
the preference assessment, one should
consider both the ecological fit and likely
effectiveness. - Separate preference assessments should
be conducted for different classes of
stimuli
Some General Recommendations
