6-fundamentals of single case experimental design ll Flashcards
Based on an AB design, but the treatment phase is DIVIDED into subphases
Each sub phase involves a DIFFERENT behavioral CRITERION i.e., a different value of the IV
CRITERION in each subphase more closely resembles the TERMINAL behavior GOAL
Changing Criterion Design: procedure
Each sub phase provides:
- A means to gauge the effects of changing the IV value from the PREVIOUS phase
- REPLICATES the effects of changing the IV value
- A baseline for the FOLLOWING phase
PREDICTION if no other change in the IV is made
Changing criterion design: logic
Experimental CONTROL is demonstrated when performance closely matches the specified criteria:
UNLIKELY that an EXTRANEOUS variable produced the change across conditions if the behavior changes when and ONLY when a NEW criterion is ??
Changing criterion design: CONTROL
Do you need perfect correspondence with criteria to show experimental control?
NO…
-Examine mean SHIFT to determine if it shows STEP-WISE changes
Examine PERCENTAGE of points that meet criterion
(Changing criterion Design: evaluation
Implement BI-DIRECTIONAL changes to BOLSTER demonstration of experimental control
- Changing criteria to a PREVIOUS subphase VALUE and observing if that behavior reverts to that criteria - /Rules out THREATS such as MATURATION and PRACTICE effects
Bi-Directional criteria… Changing criteria
Number of criterion changes
At minimum, TWO, otherwise an AB design
- Too many changes, to rapidly, may OBSCURE orderly EFFECTS
Changing criterion: number of phases
As always, determined by STABILITY (again each sub phase acts as baseline for the next phase)
Phases can be shorter if behavior can change rapidly
LENGTH of phases should VARY- an additional demonstration of control: you change or keep behavior at a given level for as long or as briefly as you can
Changing criterion: PHASE LENGTH (I.e., session per phase)
SMALL, INITIAL criterion changes maximize Probability of success, but…
If too small, why not be able to detect CHANGES IN BEHAVIOR
If too LARGE, may be difficult for the subject to
MEET CRITERIA
Rule of thumb: SMALL changes for a very STABLE behaviors, LARGER changes for VARIABLE behaviors
Changing criterion: degree of change.
SMALL changes for very STABLE behaviors, LARGER changes for variable behavior
Rule of thumb: changing criterion
Treatments do NOT have to be WITHDRAWN. (relative to reversal designs)
Does NOT require multiple behaviors, subjects, or settings (relative to multiple baseline)
All subjects can receive TREATMENTS after the SAME length of baseline
Changing criterion: advantages
Requires considerable TIME and EFFORT in planning
Phase length, degree of change, phase number should be planned in advance
Difficult to INTERPRET when behavior does NOT closely MATCH criteria
What if behavior drops to zero during first criterion change? Where is the experimental control?
Changing criterion: limitations
Use when it is meaningful to MEASURE behavior change in STEPWISE increments
E.g., NUMBER of products such as cigarettes smoked: math problems completed; connecting links in a behavioral chain
Use to demonstrate experimental CONTROL during FADING and SHAPING procedures.
Changing criterion use
Also known as:
SIMULTANEOUS treatment design – but not really simultaneous
CONCURRENT schedule design – but unlike a concurrent schedule of reinforcement
Multiple SCHEDULE design – but not necessarily like a multiple schedule
MULTIELEMENT design
Alternating Treatment Design BUT NOT Necessarily TREATMENTS
Rapid ALTERNATION of two or more IVs or LEVELS of the independent variable
Each session may be a different condition
REPEATED measurement of behavior while the conditions alternate RAPIDLY
INDEPENDENT VARIABLES continue alternating INDEPENDENT of the level of responding
No WAITING for STEADY state
Multi elemental description
Shares logical properties with the REVERSAL DESIGN (akin to A reversal design with very brief phases)
Each data point :
•PREDICTS FUTURE behavior in the same condition
•VERIFIES previous predictions
•COMPARISONS against the PREDICTIONS made by the data in the OTHER conditions
Multi element logic
Rapid comparison of TREATMENT to BASELINE
Comparison of TWO or more treatments
Comparison of two or more ASSESSMENT conditions (e.g., functional analysis of behavior)
YOKED Elements - rapid alternation makes yoking more meaningful.
—Yoking: elements from one condition are linked to elements of a second condition
Multi element: Common Uses
Elements from one condition are linked to elements of a second condition
Yoked
Plus pre-treatment baseline, preferable unless clinically contra indicated
With no baseline
•Often includes a baseline or control as one of the alternating conditions
•If so, may differ from a pre-treatment baseline due to multiple treatment interference
With baseline plus a final treatment phase
• Clinically prudent in applied research.
•Permit detection of possible multiple treatment interference
Multielement: common variations
This Multi element variation is preferable unless clinically contra indicated
Multi element plus pre-treatment baseline
This multi element variation often includes a baseline or control as one of the alternating conditions. If so, may differ from a pre-treatment baseline due to multiple treatment interference.
Can include as one of its conditions, a “no treatment “ condition, Which is essentially a baseline condition that is not the initial distinct phase but rather is alternated randomly with all the other conditions under consideration.
Multi element with no baseline
This variation is clinically prudent in applied research. Permits detection of possible multiple treatment interference
Multi element with baseline plus a final treatment phase
Each sub phase involves a different behavioral criterion i.e., a different value of the IV
Changing criterion: Procedure
Criterion in each subphase more closely resembles the terminal behavior goal
Changing criterion design
Concluding that the independent variable has produced a change in the dependent variable but in fact the relation does not exist… False positive
Type l error
Analysis of single subject data
Concluding that the independent variable has not produced a change in the dependent variable when in fact it has… False negative
Type ll error
Doing a treatment analysis, a behavioral intervention is evaluated and found effective in decreasing problem behavior. A drug intervention was arm concurrently, independent of the behavioral intervention. The drug is later withdrawn , and problem behavior increases during the behavioral treatment analysis. The initial conclusion of the behavioral interventions effect may be considered…
A type l error
Implement bi-directional changes to
Bolster demonstration of experimental control
Treatments do not have to be withdrawn wrote it to reversal designs.
Does not require multiple behaviors, subjects, or settings, relative to multiple baseline
All subjects can receive treatment after the same length of baseline
Changing criterion advantages
Subject to multiple treatment interference. Would the effects of anyone treatment be different if it wasn’t being simultaneously compared with another?
Can be examined/controlled by including a final best treatment phase.m
Unsuitable for individuals that have problems forming discriminations. Differences between conditions will appear as a function of how easily the conditions can be discriminated. I.e., how dissimilar They are
Like reversal limited to behavior that is reversible at least pliable
Less suitable for interventions that produce change slowly or require continuous implementation e.g., weight control intervention
May require considerable care in doing the necessary COUNTERBALANCING
Multielement limitations
Less of a problem if the independent variables are themselves quite different.
Providing and anchoring additional stimuli e.g., therapist, ambient stimuli, to facilitate discrimination
Reducing the number of conditions
Instructional control been appropriate
Reverting to other designs
Multi element enhance discriminability
Inclusion of elements from two or more designs within the SAME experiment.
•Enhances the certainty of experimental control if it meets the requirements of multiple designs, especially if the conclusions of the planned design are tenuous, e.g., discriminability problems in multi element, add reversal.
•Can’ answer multiple questions, e.g., is A better than BL and is B better than A?
Often not planned, but used to enhance conclusions as the data evolve. Let data be your guide
Design combinations
Systematically withdrawing treatment components to see if behavior change is maintained
Used To evaluate maintenance of treatment effects in the absence of the intervention
Used as a Fading process
Component analysis/sequential withdrawal
The systematic examination of the effects of a range of values of the IV. Not just absences versus presence of the IV
Examples, the effects of differing values of a reinforcement schedule, e.g., FR1 verses of FR 10
Comparison of treatments at different strengths, e.g., brief time out versus long time out
Parametric analysis: description
Determining effective paramedic values of contingencies, such as
Reinforcer Duration or magnitude,
reinforcer delay,
reinforcer quality,
response effort or schedule.
As such, useful for
ease of implementation questions,
treatment degradation questions
Parametric analysis use
Assessment of behavior on occasions when the contingencies arranged in the analysis are not in effect
Uses, evaluate whether treatment effects are evident before treatment occurs.
Is further training really needed?
Examples, transfer of training, multiple probe technique
Use of probes: description
Events not related to the independent variable that may affect the dependent variable.
Potential to alter the results if not controlled
Extraneous variables
An uncontrolled factor known or suspected to exert influence on the DV
Likely to have altered the results
Examples
Carryover
Sequence effect
Multiple treatment Interference
Confound
Confounds introduced by uncontrolled variables related to aspects of the environmental design.
Most common instances, (often unrelated)
Multiple treatment interference,
sequence events
, carryover events
Confounding design interactions
If subjects are exposed to multiple treatments….
- Conclusions about the outcome of each may be restricted to that specific context
- Treatments may have produced a different effect in isolation
Multiple treatment interference-CONFOUND
The effects on a person‘s behavior in One condition can be influenced by the subjects experience in a prior condition.
Do you get the same effect in the C phase of an ABCBC design if instead it has been an ACBCB design?
Sequence effects
CONFOUND
PATTERNS of behavior established in one session…
• May inadvertently extend into a second session.
• Even if the independent variables are very different….
•Calling into question the observations in the subsequent session, (INFLUENCES of the INDEPENDENT VARIABLE or influence of the IV in the prior session?)
Carry over effects
CONFOUND
Manipulate only one IV at a time
Counterbalance the design :
E.g., if subject A starts with treatment A First, let subject B start with treatment B first. • ALTERNATE order of conditions Within subjects
ENHANCE DISCRIMINABILITY
When multiple treatments COMPARED, end it with a final evaluation in ISOLATION
Minimize Confounding Variables
All data analysis methods strive to minimize these Errors but vary with respect to the emphasis on each.
Visual inspection – minimize type one error, perhaps at the Expense of making more type ll error’s
Test of statistical significance – minimize type to error’s, perhaps at the expense of making more type one error’s
Minimizing type one and two errors
In behavior analysis, we seek robust variables that produce large effects: the criteria for concluding an effect using visual inspection are very stringent
To the point we are visual inspection may disregard actual differences or changes when they are real but small
Small often equals not clinically significant
Type l error and robust effects
More likely to identify independent variables that produce robust Effects
Social significance is of primary importance: statistical versus clinical significance
Encourage the examination of variability rather than just overall effects
Advantages of visual inspection
Factors involved in making data decisions based on visual analysis of single subject data
Analysis of single subject data
Mean/level,
trend,
latency to change,
other factors
Changes in the average level of performance within a phase or condition.
The greater this shift relative to the Comparison condition, the more convincing the effects
Visual data analysis: MEAN SHIFT
changes in the tendency for the data to increase or decrease over time:
Does the data in the preceding phase predict something different than what occurred in the test phase?
Was the trend in the preceding phase consistent with the conclusion made about the test phase?
Visual data analysis: TREND
Latency to change – how quickly does the behavior change once the independent variable is manipulated?
Shorter equals more convincing the effect.
Long to change produce suspicion
Visual data analysis: LATENCY
Variability and overlap: e.g., how much do the data points during intervention overlap with the data points in baseline?
Phase duration: e.g., was the change in intervention demonstrated long enough to convincingly show it was different from baseline
Consistency of the effect in replication: e.g., does the level revert to similar levels if you return to the intervention phase a second time
Visual data analysis: other factors
Do we really need it? Counter argument: independent variable in accuracy would be immediately detectable through changes in the dependent variable. But changes may not occur even if Independent variable is implemented in accurately
Extraneous variables my account for a steady state
Procedural integrity measures: necessary?
Often measured in the same way as
The dependent variables, measurement of procedural reliability.
Measures the extent to which the application of the IV over the course of an analysis matches the plan description.,
E.g., the reinforcer was delivered within five seconds on 98% of the occasions for which it was scheduled.
Provides the experimenter with data regarding whether calibration of the treatment agent is needed
Procedural integrity measurement
Placing a high priority on independent variable simplification. Elimination complexity and ambiguity
Provide adequate training and practice for those implementing treatment. Curtail procedural drift
Direct contingencies on treatment Fidelity. Not just measurement, but intervention to promote procedural integrity
Reducing procedural integrity threats
Other considerations. Be a critical consumer of research, approach methods, conclusions skeptically
Be a consumer of broth sorts of research. Your interpretation may differ from those in other disciplines… But good data is good data, reliable effects are we liable effects.
It May help to consider what sorts of things get manuscripts rejected
Being a research consumer
Description: examination of the acceptability or viability of a program intervention
IE, are the changes in behavior of clinical or applied importance
Social validity assessment
Social significance of goals or target behavior.
Represents deficit in functioning as society views it.
E.g., lack of appropriate social skills.
Will increase or decrease in the measured dimension of the behavior result in improvement in a persons life
Social validity: of goals and targets
Appropriateness of the procedures: one that produces minimal adverse affects e.g., a drug reduce his aggressive behavior without producing gross sedation
One and that can be practically administered: complexity, practicality, cost
Regardless of possible effects, Unacceptable treatment variables will not be used
Social validity: of procedures
Social importance of the results
:
Enhances subjects functioning in their environment
E.g., disabled person becomes competent in the use of public transportation.
Ultimately, is the person better off now that the behavior has changed. Well it was old and increased opportunity for reinforcement
Social Validity: of results
Methods:
• client, those important in their life, experts, ….EVALUATE whether distinct improvements have been achieved
AND
•is the change is worth the cost and effort
-• use of rating scales
Social Validity: subjective evaluation
My expectations for a satisfactory outcome of the training program is,
• 1 =very pessimistic, 7 equals very optimistic
I feel the approach to infant care by using this type of training program is…
• 1 equals very appropriate: 7 equals very appropriate
At this point, I think my husband’s ability to handle care taking concern is, 1 equals considerably worse 7 equals greater improved
( Rating Scales)
Social validity: sample questions
Consumer evaluation may be inconsistent with actual changes in behavior
High levels of satisfaction do not necessarily indicate there has been an important change
Who actually constitutes the consumer?
Subjective evaluation considerations
Is the behavior after treatment comparable to unaffected or normal peers
• Identify RELEVANT peers based on important characteristics e.g., age gender, SES, but different with respect to the target behavior
•Identify whether the individual was brought to within comparable parameters regarding the relevant behavior
Social validity: social comparison
Gajar 1984 -conversation skills in youth with head trauma
•Confabulatory and perseverative responding
Inability to stay on topic
Excessive self disclosure, interruptions
Inappropriate laughter
Peer group
Groups of 20 to 22-year-old college students.
Provided a topic for conversation
. Rate of positive conversation or behaviors.
Social comparison
does the level revert to similar levels if you return to the intervention phase a second time
Consistency of the effect in replication: Visual data analysis
how much do the data points during intervention overlap with the data points in baseline?
Variability and overlap: Visual analysis
was the change in intervention demonstrated long enough to convincingly show it was different from baseline
Phase duration: Visual analysis
Useful for highly variable behavior that fluctuates as a function of non-experimental variables
Multielement Advantages
Can absorb the influence of extraneous variables as long as clear and consistent differences remain between conditions.
Can be more efficient, in terms of the number of sessions, then longer designs
Muti- element advantages
Used To evaluate maintenance of treatment effects in the absence of the intervention
Used as a Fading process
Component analysis/sequential withdrawal
Most common instances, (often unrelated)
Multiple treatment interference,
sequence events
, carryover events
Confounds introduced by uncontrolled variables related to aspects of the environmental design
Each data point predicts future behavior in the same condition
Each data point serves to verify previous predictions
Each data point permits comparison against the predictions made by data in the other conditions
Multielement logic
Experimental control is demonstrated when behavior is appreciably and consistently different in one condition relative to others
Multielement logic
Remember on graph if you see larger changes in the criteria that means there was greater variability in the previous trial. If there are smaller changes then there is less variably or it is more stable
Changing criterion graph
Lower the criteria at some point during the trials in order to demonstrate experimental control
Bi- directional criteria graph
an experimental design consisting of an initial baseline phase, an intervention phase, and a return to baseline conditions by withdrawing the independent variable to see whether responding “reverses” to levels observed in the initial baseline phase.
ABA Reversal Design
An experimental design in which first one element of the treatment is withdrawn, then a second, and so on, until all elements have been withdrawn; particularly well suited to assessing behavior for maintenance.
Sequential Withdrawal design
A systematic assessment of 2 or more independent variables or components that comprise a treatment package. Component analyses are important for the analysis of behavior; however, previous research provides only cursory descriptions of the topic.
Component analysis
; J. O. Cooper, Heron, & Heward, 2007). Researchers and clinicians conduct component analyses to identify the active components of treatment packages that are responsible for behavior change. For behavioral treatments to be analytic, researchers must identify specific components of a treatment package that produce behavior change (Baer et al.). Component analyses also may enhance the efficiency and social validity (Wolf, 1978) of behavioral treatments by eliminating ineffective and perhaps effortful components and by evaluating the necessity of more restrictive components (e.g., punishment procedures) or those components of the intervention that are unnecessary. This in turn may lead to better generalization and maintenance of the program if parents, teachers, or staff have to be trained on only the key elements of the intervention. Finally, conducting a component analysis is a skill required of Board Certified Behavior Analysts
Component analysis
Used to demonstrate experimental control during fading and shaping procedures.
Changing criterion uses
Strict alternation
Research answe
The utility of this design relies on stimulus DISCRIMINATION
•. MIXED vs MULTIPLE schedules.
•. Best it each condition associated with a distinct set of stimuli to promote discriminability
Multi element: discriminability
Conditions are counterbalanced ACROSS experimental CONTEXTS such as time of day, therapist, to neutralize confounding factors.
• Example, if treatment A always run in the morning and treatment B in the afternoon, and differences emerge… you don’t know if the effects are attributed to the treatments or to the time of day
• Counterbalance by someone running A in the morning and at other times, running A in the afternoon.
Multi element: counterbalancing
Strict alternation not recommend , as it does not neutralize sequence effects
Randomization
• NOT ABABABAB
• Rather ABBABAAABBABB
Randomization with restriction
• E.g. no more than two of the same condition in a row
Multi element: order of conditions
Compare treatments while minimizing sequence effects of differential results stemming from the order of implementation of independent variables. Can occur in reversal designs because each independent variable must be in effect for a long period of time.
Minimizes these effects through random alternation.. sometimes A follows B, sometimes B follows A.sometimes B follows B. And short periods in which each IV is in effect
Useful for highly variable behavior that fluctuates as a function of non-experimental variables
Can’t absorb the influence of extraneous variables as long as clear and consistent differences remain between conditions.
Can be more efficient, in terms of the number of sessions, then longer designs
Multi element advantages
an experiment designed to discover the differential effects of a range of values of an independent variable.
Parametric
Experimental control is demonstrated when behavior is APPRECIABLY and consistently DIFFERENT In one condition RELATIVE to others.
Multielement- logic
Plus pre-treatment baseline, preferable unless clinically contra indicated
Multi elements – common Variations
The extent to which the independent variables are implemented as dictated by the research or treatment plan.
Can be a major source of confounding variables
Inconsistencies among therapist can influence data
Procedural drift over time may change behavior in the absence of a planned change in the independent variable
Procedural integrity assessment
We already knew that.
Is this finding actually novel?
Does it contribute significantly beyond what we already knew
Understanding, prediction, and control
Contributes to the applied realm
Is this about behavior? Or is this about a hypothetical construct?
You can’t convince me these results aren’t caused by something else. Extraneous variables and confidence. The design did not permit a reasonable demonstration of experimental control
I don’t think your dad I sufficiently convincing to support those conditions.
Effects insufficiently robust
Poorly describe a methodology
Is this meaningful or useful outside of the tightly controlled confines of the laboratory
Consuming research
Other considerations:
Be a critical consumer research; approach methods, conclusions skeptically
Be a consumer of broad sorts of research
Your interpretation may differ from those in other disciplines… But good dad is good data:
I may help to consider what sorts of things get manuscripts rejected…
Being a research consumer
Under some circumstances, the most direct test of social validity can be accomplished by permitting the person to select directly from treatment options
* assuming all interventions are equally effective * Can we use even if limited expressive language
Social validity consumer choice
Alternate order of conditions within subjects
Example, If subject A starts with treatment A first, let subject B start with treatment B first.
Example of counterbalance the design, to minimize con found
