Experimental Design Flashcards
When a PREDICTABLE CHANGE in behavior (dependent variable or DV) can be reliably produced by the SYSTEMATIC MANIPULATION of some aspect of the individual’s environment (independent variable or IV).
The ANALYSIS dimension of the 7 dimensions of ABA.
AKA: Functional Relations; Control; Analysis
Experimental Control
4 Important Elements of Behavior
- ) Behavior is INDIVIDUAL (one person’s interaction with the environment)
- ) Behavior is CONTINUOUS (changes overtime)
- ) Behavior is DETERMINED (by functional relations it holds to other events)
- ) Behavior variability is EXTRINSIC to the organism (i.e., behavior change is the result of the environment; the IV, other people and/or uncontrolled factors)
6 Components of Experiments in ABA
- ) At least one Subject
- ) At least one Behavior (DV)
- ) At least one Setting
- ) At least one Treatment (IV)
- ) A measurement system and ongoing analysis of Data
- ) An experimental Design
All well-planned experiments begin with this.
A brief but specific statement of what the researcher wants to earn from conducting the experiment.
Can be in question or statement form.
Experimental Question
In this design the subject acts as his or her own control; does not mean that there is only one subject.
AKA: Single-case designs; Within-Subject Designs, Intra-Subject designs
ABA uses:
Single-Subject Designs
In this design repeated measures of the subject’s behavior during each phase of the study provide the basis for comparing experimental variables as they are presented or withdrawn in subsequent conditions (the presence and absence of IV).
The individual is exposed to each condition several times over the course of the study.
At least one subject: (Single Subject Designs)
Collateral Effects
A phenomenon in which the IV effects behaviors other than the targeted behavior.
Provide data patterns that can serve as controls for evaluating and replicating the effects of an IV.
Assess if any collateral effects occur.
Determine whether changes in the behavior of a person other than the subject occur during the course of an experiment and if such changes can explain changes in the subject’s behavior.
At least one behavior: (AKA: Dependent Variable - DV)
Control 2 sets of environmental variables to demonstrate experimental control:
- ) IV (present, withdraw, or vary its value)
- ) Extraneous Variables (prevent unplanned environmental variation)
Better to control in labs but in applied settings (homes, schools etc., it is harder to control the environment).
At least one (1) Setting
The particular aspect of the environment that the experimenter manipulates to find out whether it affects the subject’s behavior.
AKA: Independent Variable (IV); Intervention; Experimental Variable
At least one (1) Treatment
Observation and recording procedures must be conducted in a standardized manner.
Standardization involves every aspect of the measurement system ( from the behavior definition to scheduling of observations).
Behaviorists must detect changes in level, trend, and variability.
A measurement system and Ongoing analysis of Data
The particular arrangement of conditions in a study so that meaningful comparisons of the effects of the presence, absence, or different values of the IV can be made.
Experimental Design
The value of the IV is manipulated. Seeks to discover the differential effects of a range of values.
Ex. Various doses of medication are given in a course of a study.
Parametric Analysis
Two types of Experimental Designs:
- Nonparametric Design
2. Parametric Design
IV either present or absent during study.
HINT: Has the word ON in it (IV is either ON or OFF)
Ex. Medication is either given and then taken away in the course of a study.
Nonparametric Design
- Change only one variable at a time.
- Do not get locked into textbook designs.
- Select and combine designs that best fit the research question.
Important Rules of Experimental Designs
AKA: Behavioral Package
When multiple IVs are bundled into one program such as a token economy with praise and time-out.
Treatment Package
Looks at the effect of each part of the treatment package.
Used to determine the effective components of an intervention package.
Component Analysis
AKA: Stable State Responding
A pattern of responding that exhibits very little variation in its measured dimensional quantities over a period of time.
Provides the basis for BASELINE LOGIC.
Steady State Responding
Refers to the experimental reasoning inherent in single-subject experimental designs.
Entails 3 elements:
- ) Prediction
- ) Verification
- ) Replication
Each of these elements depends on an overall experimental approach called steady state strategy.
Baseline Logic
REPEATED EXPOSURE of a given condition while trying to eliminate extraneous influences on behavior and obtaining a stable pattern of responding before introducing the next condition.
Steady State Strategy
FUNCTION of Baseline Data
Serves as a control condition.
Does NOT imply the absence of intervention. It can be the absence of a specific IV.
BENEFITS of Baseline Data
- To use the subject’s performance in the absence of the IV as an objective basis for detecting change.
- To obtain descriptions of ABC correlations for the planning of an effective treatment.
- To guide us in setting the initial criteria for reinforcement.
- To see if the behavior targeted for change really warrants intervention.
4 Patterns of Baseline Data:
Hint: DAVS
D- Descending Baseline
A- Ascending Baseline
V- Variable Baseline
S- Stable Baseline
Descending Baseline:
Shows the behavior is already changing.
Generally, one should NOT implement the IV when baseline is descending; unless it is a functional skill that you are trying to increase and the decreasing trend shows the behavior worsening.
If descending baseline is due to a behavior you want to decrease you should wait because behavior is already improving.
Ascending Baseline:
Shows the behavior is already changing.
Generally, one should NOT implement the IV when baseline is ascending; unless it is a challenging behavior that you are trying to decrease and the increasing trend shows it is worsening.
If ascending baseline is due to a behavior you want to increase you should wait because behavior is already improving.
Variable Baseline
No clear trend
If one’s data is variable, wait it out and do not introduce the IV.
Variability is assumed to be due to environmental variables that are uncontrolled. If the IV is introduced here, you will not be able to tell if it changed the behavior or not.
You should try to control uncontrolled sources of variability.
No evidence of ascending or descending trend.
All of the values of the DV fall in a small range of values.
BEST way to look at the effects of the IV on the DV.
You can introduce the IV now.
Stable Baseline
3 Parts of Baseline Logic:
Hint: PVR
Prediction
Verification
Replication
After we PREDICT we:
AFFIRM
Inductive Logic:
- If the IV were not applied, the behavior (as indicated by baseline data would not change).
- The experimenter predicts the IV will change the behavior.
- If the IV controlling the DV (A), then the data path in the presence of the IV will show that the DV (B) has changed.
- When the IV is present, data show DV has changed (B is true)
- Thus, the IV is controlling the DV (thus, A is true).
Affirmation of the Consequent
The anticipated outcome of a presently unknown measurement.
Data should be collected until stability is clear.
Main question: Are data stable enough to serve as the basis for experimental comparison?
Prediction
Verifying a previously predicted level of baseline responding by termination or withdrawal of the treatment variable.
Verification
Is the essence of believability.
Shows reliability of behavior change; we can make it happen again.
Is accomplished by reintroducing the IV.
Replication
5 Main Experimental Designs
HINT: MC RAW
- Multiple Baseline
- Changing Criterion
- Reversal
- Alternating Treatments
- Withdrawal
- MOST WIDELY USED design.
- Highly flexible.
- Staggered implementation of the intervention in a step-wise fashion across BEHAVIORS, SETTINGS, & SUBJECTS.
- Do not have to withdraw a treatment variable in this design.
- When it is UNETHICAL or impractical to reverse conditions or when the behavior is irreversible use this design instead of a reversal design.
Multiple Baseline Design
A functional relation in this design requires change in behavior with the onset of the intervention.
Apply IV to behavior 1 when you can confidently predict that the behavior would remain the same in constant conditions.
If behaviors 2 & 3 remain unchanged after the application of the IV to behavior 1 , this verifies the prediction.
If the IV changes behavior 2 like it did behavior 1, the effect of the IV has been replicated.
The more replications, the more convincing the demonstrations.
Most commonly 3 to 5 tiers
Prediction, Verification and Replication in the Multiple Baseline Design
(How to Demonstrate Functional Relations with this Design)
Two or more different behaviors of the SAME SUBJECT.
Each subject serves as his/her own control.
After steady state baseline responding, the IV is applied to the first behavior while other behaviors are kept in baseline.
When steady state responding is reached for the first behavior, then the IV is applied to next.
Multiple Baseline Across Behaviors
A single behavior is targeted in two or more different settings or conditions.
After steady state baseline responding, the IV is applied to the first setting while other settings are kept in baseline.
When steady state responding is reached for the first settings, then the IV is applied to the next setting.
Multiple Baseline Across Settings
One target behavior for 2 or more subjects in the SAME SETTING.
After steady state baseline responding, the IV is applied to the first subject, while other subjects are kept in baseline.
When steady state responding is reached for the first subject, then the IV is applied to next subject.
MOST WIDELY USED MULTIPLE BASELINE GRAPH.
Multiple Baseline Across Subjects
2 Variations of Multiple Baseline Designs:
Both inherently weaker than traditional multiple baselines.
Can be used when extended baseline measurements is unnecessary, impractical, too costly or unavailable.
- ) Multiple Probe Design
2. ) Delayed Multiple Baseline Design
Analyzes relation between the IV and acquisition of skill sequences.
Instead of simultaneous baselines, probes provides the basis for determining if behavior change has occurred prior to intervention.
Multiple Probe Design
Initial baseline and intervention begin and subsequent baselines are added in a delayed or staggered fashion.
Effective when 1) reversal design is not possible, 2) limited resources preclude a full-scale design, and 3) when a new behavior, subject, or setting becomes available.
Limitation: Shorter baselines do not show interdependence of DVs.
Delayed Multiple Baseline Design
Guidelines For Multiple Baseline Design:
- ) Select independent, yet functionally similar baselines. (behaviors are functionally independent and share enough similarity, that changes will occur from the same IV)
- ) Select concurrent and plausibly related multiple baselines.
- ) Do not apply the IV to the next behavior too soon.
- ) Vary significantly the lengths of multiple baselines. (the more variation, the stronger the design)
- ) Intervene on the most stable baseline first.
Advantages of Multiple Baseline Design:
- Successful intervention does NOT have to be removed.
- Evaluates generalization.
- Easy to implement.
Disadvantages of Multiple Baseline Design:
- ) Functional relationship is NOT directly shown in this design.
- ) Effectiveness of the IV is demonstrated, but not information regarding the function of the target behavior.
- ) IV may be delayed for certain behaviors, settings, or subjects.
- ) Takes resources to implement properly.
Experimental design in which an initial baseline phase is followed by a series of treatment phases consisting of successive and gradually changing criteria for reinforcement or punishment.
Technically, it is a variation of the multiple baseline design.
Changing Criterion Design
- ) The criterion lines should have a large separation to show a functional relationship.
- ) Experimental control is evidenced by the extent that the level of responding changes to conform to each new criterion.
- ) If data points do not fall around the criterion lines, that shows us that there is very little experimental control.
- ) The greater the vertical distance between the criterion lines, the more experimental control.
Prediction, Verification, and Replication in the Changing Criterion (How to Demonstrate Functional Relations with this Design)
There is only ONE behavior in this design.
Behavior in this design has to already be in the subject’s repertoire.
Evaluates treatment that is applied in a graduated or step-wise fashion.
Changing Criterion Design
- ) Length of phase
- Each phase must be long enough to achieve stable responding.
- Target behaviors that are slower to change require longer phases.
- Validity of the design is increased when you vary the length of each phase. - ) Magnitude of criterion changes
- The size of the changes between each criterion should vary to prove functional relations.
- Changes in size must be large enough to be detectable, but not so large as to be unachievable.
- Changes in size can be similar if you are dealing with stable data. - ) Number of Criterion changes
- The more criterion changes the better proof of experimental control.
Guidelines for Changing Criterion Design
Advantages of Changing Criterion Design
- Does not require reversal of improved behavior.
2. Enables an experimental analysis within the context of a gradually improving behavior.
Disadvantages of Changing Criterion Design
- The target behavior must already bein the person’s repertoire.
- Not appropriate for analyzing the effects of a shaping program.
- It is NOT a comparison design.
MOST POWERFUL WITH-IN SUBJECT DESIGN for demonstrating function.
Preferred over A-B-A as stronger design.
AKA: A-B-A-B; B-A-B
Requires at least 3 consecutive phases:
Initial baseline (A)
Intervention (B)
Return to Baseline (A)
Reversal Design
Involves prediction, verification, and replication.
The IV is responsible for behavior change if repetition of baseline and treatment phases approximate the original phases.
Solid data points= Actual measure of behavior
Open data points= Predicted data if conditions from previous phase remained in effect
Data in shaded Box in Baseline 2= Verification of prediction from baseline 1.
Data in Cross-Hatched Shaded Box= Treatment data replicating the experimental effect
Prediction, Verification, and Replication in the Reversal Design (How to demonstrate Functional Relations with this Design)
Any experimental design in which the researcher REVERSES responding to a level obtained in a previous conditioned.
Encompasses experimental designs in which the IV is withdrawn (A-B-A-B) or reversed in its focus (e.g., DRI/DRA).
Alternation between baseline and a particular intervention.
Each reversal strengthens experimental control.
Evidence of a functional relation is strengthened with each reversal
For a reversal to occur the behavior must approximate the initial baseline level
Reversal Design
If your client is displaying severe and dangerous behavior (SIB, elopement), then do NOT spend time just taking baseline data from the start. Is your ETHICAL responsibility to start treatment immediately.
Use this type of reversal design:
B-A-B Reversal
5 Variations of the Reversal Design
- Repeated Reversal
- B-A-B Reversal
- Multiple Treatment Design
- NCR Reversal Technique
- DRO/DRI/DRA Reversal Technique
3 Phase reversal design: IV, IV removed, IV reintroduced
Weaker than the A-B-A design because it does not enable assessment of the effects of the IV during baseline.
Disadvantage: Sequence effects because the level of behavior in condition A may have been influenced by the IV before it.
Best design to use when your client displays dangerous and severe behaviors; also appropriate for when an IV is already in place and you have limited time.
B-A-B Reversal Design
Effects on a subject’s behavior in a given condition that are the result of the subject’s experience with a prior condition.
AKA: Carryover Effects; Alteration Effects
Sequence Effects
An experimental technique for showing the effects of reinforcement by using NCR as a CONTROL condition INSTEAD of a baseline condition in which no reinforcement is provided.
Allows us to examine contingent reinforcement.
The reinforcer is presented on a fixed or variable time schedule independent of the subject’s behavior.
NCR Reversal Technique
A type of reversal design that compares 2 or more IVs to baseline and/or one another.
Reversal design that has multiple letters added (e.g., A-B-A-C-A-B-A-C, A-B-C-D-A-C-A-D)
Disadvantage: Sequence Effects
Multiple Treatment Reversal
An experimental technique for showing the effects of reinforcement by using DRO, DRA, or DRI as a CONTROL condition INSTEAD of a baseline condition in which no reinforcement is provided.
Allows us to examine contingent reinforcement.
DRO/DRA/DRI Reversal Technique
Advantages of Reversal Design
- ) Clear demonstration of the existence or absence of a functional relation between the IV and DV.
- ) Enables us to count the amount of behavior change.
- ) Return to baseline tells us we need to program for maintenance.
Disadvantages of Reversal Design
- Irreversibility
2. ETHICAL ISSUES, as well as social and educational issues can arise when you REMOVE an effective IV.
The level of behavior observed in an earlier phase cannot be reproduced even though experimental conditions are the same as they were during the earlier phase.
When _________ is a problem, use DRO/DRI/DRA conditions as control techniques or multiple baseline designs.
Irreversibility
AKA: Simultaneous Treatments Design; Concurrent Schedules Design; Alternating Treatments Design; Multi-element Baseline Design; Multi-Element Design; Multiple Schedules Design
Acronym: SCAMMM
Alternating Treatments Design
An experimental design in which 2 or more conditions are presented in rapidly alternating succession independent of the level of responding and the differential effects on the target behavior are noted.
Alternating Treatments Design
Compares 2 or more IVs to one another to see which IV would be best to utilize with a client.
Based on stimulus discrimination (each IV has an obvious Sd signaling which IV is in effect at any given time)
Data plotted separately on the same graph.
IVs may be:
- Alternated across daily sessions
- Given in sessions occurring the same day
- Implemented during each portion of the same session
Alternating Treatments Design
3 Variations of the Alternating Treatments Design:
- ) Single Phase Without Baseline: Does not require an initial baseline.
- ) With Baseline: Whenever possible, baseline should be conducted, as it shows the change produced by each treatment compared to the natural level of performance without an intervention.
- ) With Baseline and Final Best Treatment Phase: Most widely used.
On Graphs: Visual inspection of the difference between or among the data paths produced by each treatment.
Functional relation shown when:
- One data path is consistently higher than the other.
- No overlapping data paths.
The degree of differential effects produced by 2 different treatments is determined by the vertical distance between the respective data paths.
Prediction, Verification, Replication: Not identified in separate phases of the design.
Each successive data point in treatment plays all 3 roles.
Prediction, Verification, & Replication in Alternating Treatments Design (How to Demonstrate Functional Relations with this Design)
3 Problems Avoided by Alternating Treatments Design
HINT: ISU
- Irreversibility
- Sequence Effects
- Unstable Data
- Does not require treatment withdrawal
- Speedy comparison
- Minimizes irreversibility problem
- Minimizes sequence effects
- Can be used with unstable data
- Can be used to access generalization of effects.
- Intervention can begin immediately without baseline data.
Advantages of Alternating Treatments Design
- Multiple Treatment Interference: this is always a problem with this design, as multiple treatments are going on at the same time.
- Unnatural nature of rapidly alternating treatments.
- Limited capacity of the design (suggested maximum comparison of 4 conditions, although more have been reported in research)
- Selection of treatments: Should be significantly different from 1 another.
Disadvantages of the Alternating Treatments Design
Used to describe experiments based on A-B-A-B analysis.
Also used to describe experiments in which an effective treatment is sequentially or partially withdrawn to promote the maintenance of behavior changes.
Withdrawal Design
How To Identify Practical and Ethical Considerations in Single-Case Experimental Designs to Demonstrate Treatment Effectiveness
Your # 1 goal when using single-case designs is to clearly show that your IV changed the target behavior and nothing else.
It is your job to identify the the practical and ethical issues about single-case designs showing treatment effectiveness. 3 Practical and ethical issues of concern are:
- Baseline Trends
- Excessive Variability in Data
- Duration of Phases
- Increasing or decreasing trends in your data during baseline data collection do NOT allow you to clearly demonstrate that your IV caused the change in behavior.
- How to address: continue observations for a longer time period; try to reverse the trend (use DRO schedule); select designs that do not require a stable baseline; use statistical techniques that take initial trends into account.
How to identify practical and ethical issues: (Baseline Trends)
Variability in your data can obscure intervention effects.
-How to address: block consecutive data points and plot blocked averages rather than day-to-day performance; search for causes of the variability or the situation (e.g., variation among the environmental stimuli)
Hot identify practical and ethical issues: (Excessive Variability in Data)
- The duration of each phase in your design can involve problems related to trends and variability in the data.
- How many data points at minimum do you need in each phase of your design?
- Use this for for deciding when to shift phases. this helps to reduce any subjectivity you may have about when to shift phases.
How to identify practical and ethical issues: (Duration of Phases)
Other Concerns about Single-case Designs
- The range of questions about intervention effects that can be addressed with these designs: (BEST for treatment package evaluation).
- The generality of the research results:
- Is the findings generalized beyond the subject in the design?
- To assess generality, use replication of your IV across subjects, etc.
Why does ABA have a Problem with Traditional Psychology’s Group Approach to Research?
3 Reasons:
- Group data not representation of individual performance.
- Group data masks variability (hides variability that occurs between and within subjects).
- Absence of Intra-subject Replication (power of replicating effects with individuals is lost).
The extent to which an experiment shows convincingly that changes in behavior are a function of the IV and not the result of uncontrolled or unknown variables.
HINT: = Independent Variable
Internal Validity
2 Types of Validity in Experimental Designs
Internal Validity & External Validity
An ____________ study involves only 1 IV at a time. Multiple IVs are not confounded (presented at the same time). This is the best way to see the effect of the IV on the DV.
High _________ = Designs showing strong experimental control.
Internally Valid/Internal Validity
4 Confounding Threats to Internal Validity: Hint - MISS
- ) Measurement Confounds (observer drift, reactivity, observer bias)
- ) IV Confounds
- ) Subject Confounds (maturation)
- ) Setting Confounds ( bootleg reinforcement)
Measurement confounds may occur due to:
- Observer Drift
- Reactivity
- Observer Bias (expectations)
Refers to the number and the intricacy of the behaviors you are targeting. If you are targeting numerous complicated behaviors, your internal validity may be affected.
Measurement Confounds
Observer Drift
When observers unknowingly alter the way they apply a measurement system.
This can refer to the behavior of our clients changing when observed. It can also refer to observers being affected by their data being monitored.
Reactivity
How to reduce reactivity:
Maintain baseline conditions long enough for reactivity to run its course.
The observer’s expectations that change follow in a particular direction.
Can reduce by keeping observers naive to expected outcomes of a study.
Observer Bias (Expectations)
IVs are complicated and given together usually in a treatment package.
Can be reduced by using placebo control or double-blind control procedures (in which the subject is not aware if the IV is present or not).
IV Confounds
Subject Confounds
Maturation
Repeated measurement detects uncontrolled variables.
Changes in subject over course of study.
Maturation
- Studies in natural settings are more prone to confounding variables than in controlled laboratories.
- You should hold all possible aspects of the study constant until repeated measurements again reveal stable responding.
- BOOTLEG REINFORCEMENT may occur in the natural environment.
Setting Confounds
Bootleg Reinforcement
Secretive reinforcement that is not part of your behavior plan.
Variables that exert an uncontrolled influence on a research study.
The effects of all these variables should be reduced or eliminated as much as possible in order to demonstrate experimental control.
Confounding Variables (AKA: Extraneous Variables; Unrelated Variables)
Any aspect of the ENVIRONMENT that must be held in constant to prevent unplanned environmental variation.
Ex. Lighting, space, temperature of the room
Extraneous Variables
Any uncontrolled factor known or suspected to exert influence on the dependent variable.
Confounding Variables
Degree to which a study’s results are generalizable to other subjects, settings, and/or behaviors.
Degree to which a functional relation discovered in a study will hold under different conditions.
Is on a spectrum ranging from little to a lot.
Replication establishes _______________.
External Validity
2 Major types of Scientific Replication Methods Used In ABA:
- ) Direct Replication:
- Researcher exactly duplicates a previous study.
- Intra-subject direct replication = same subject used.
- Inter-subject direct replication = different subject used. - ) Systematic Replication
- Researcher purposefully varies 1 or more aspects of an earlier experiment.
- Demonstrates reliability and external validity by showing the same effect can occur under different conditions.
- ABA research generally uses systematic replication.
AKA: Procedural Fidelity; Fidelity of Implementation; Program Integrity
Treatment Integrity
When application of the IV in later phases differs from the original application.
Treatment Drift
Extent to which the IV is implemented or carried out as planned.
Low _____ _______ makes it very difficult to interpret experimental results.
Treatment Integrity
How to Ensure A High Level of Treatment Integrity
- Precise operational definition of treatment procedures.
- Simplify, standardize, and automate, as simple treatments are more likely to be consistently delivered and simple, easy-to-implement techniques are more likely to be used and socially validated.
- Training and practice for individuals who will conduct the experimental sessions (providing a detail script, verbal instructions, etc.).
- Collect treatment integrity data to measure how the actual implementation of the conditions matches the written methods.
- Observation and calibration give the researcher the ongoing ability to use retraining and practice to ensure high treatment integrity.
- Reduce, eliminate, or identify the influence of potential confounding variables.
Assessing Treatment Integrity
Errors may be made when conducting ABA research. There are 2 main types of errors:
- ) Type I Error
2. ) Type II Error
Type I Error (AKA: False Positive)
Assuming the IV affected the DV, when it actually did NOT do so.
Statistical analysis tends to lead to more Type I errors.
AKA: False Negative
Assuming the IV did NOT affect the DV, when it actually did.
Visual analysis used in ABA studies tend to lead to more:
Type II Error(s)
