Factorial Design Flashcards
Describe a Factorial design:
- More than 1 independent variable, i.e., more
than 1 factor (grouping variables)
Describe this Factor Design:
Two Factor Design:
2 factor = 2 independent variables,
regardless of number of levels in each factor
* Described as number of levels x number of levels
2x2
2x3
3x3
* Analysed using 2-way (factor) ANOVA
Describe this Factor Design:
Three Factor Design
3 factor design would have 3 independent
variables
* 2x2x2, 3x2x2, etc.
* Analysed using 3-way (factor) ANOVA)
Independent variables are all_____________________
– IV are all independent
E.g., Do classical and rock singers show different rates of vocal polyps and does this vary with training or no voice training.
4 gpr: classical with, classical without, rock with, rock without
Describe the IV of within repeated measure:
- IV is dependent
E.g., Which hearing aid is preferred and does this make a difference with a trial period
Each participant tested on three h.a. immediately and after a month’s trial – both within (aid type - 3 levels and time - two levels)
Give an example of a Mixed Design
– Both between and within
E.g., Does the ‘Marvelous Metaphors’ program improve children’s comprehension of metaphors
1 grp gets program, 1 grp does not (between factor)
Test pretreatment, post-treatment, maintenance (within factor, 3 levels)
What are marginal means?
- Means for each independent variable (averaged across cell)
- 4 in a 2x2 factorial design
- Main effect
Difference between levels of 1 independent variable
What are Cell Means?
- Means for each cell
- Interaction effect: combinations of independent variables
What are the steps in 2-way (factor) ANOVA (between subjects)?
- Different participants in each cell
- Each measured once for DV
- Statistic
3 F-ratios
Main effect of IV1
Main effect of IV2
Interaction - Determine if p value less than alpha
- Post-hoc testing if more than 2 levels for any IV-
- Post-hoc testing if a significant interaction
- Effect sizes reported for significant differences
How would you set up this experiment:
Do the program stream and home province make a difference in satisfaction among SCSD students?
- Use a continuous scale to measure satisfaction and measure distance in mm-equal intervals.
Unsatisfied _____________ Satisfied - This is a 2-way ANOVA (between subjects)
- Split Program in HCD (2 levels) –main effect
SLP vs Audio – 2 means - Split Home (Atlantic vs ON/PQ vs Western Prov) – main effect
Atlantic vs ON/PQ vs West – 3 means
- Need post-hocs - Interaction
SLP Maritime; Audio Maritime; SLP ON/PQ; Audio ON/PQ; SLP West; Audio West – 6 means
Need post- hoc tests
In this example: 2 groups with different types of dementia are treated using 2 different interventions
How many IV groups?
What is DV?
What posthoc comparisons are interesting to test?
4 independent grps
Gain scores are analyzed (DV) using a 2-way (factor) ANOVA
4 comparisons
4 means involved
What is the Interaction Effect? (4)
- Effect of one variable not constant across different levels of the second variable
- Interaction between IVs has unique effect
- With an interaction effect, significant main effects should not be interpreted directly.
- Can still have significant main effects even if have interaction (but not likely)
What are the differences between group vs. single-subject designs?
- Both important: convergence of evidence
- Both can be experimentally rigorous
- Group designs allow greater generalizability to the larger population as a group
- Single-subject designs provide detailed analysis of individual performance
Help isolate characteristics that influence behavior
Group averages never mirror individual behavior exactly
No individual matches the ‘theoretically average” client
Does allow ‘case-to-case’ generalization
‘client-centred’ practice clinically calls for individualized care - Group designs test infrequently – can’t see natural variability and growth in measure
- Can co-occur in the same study
When should we use a single-subject design?
- When withholding treatment is considered unethical
But note treatment is delayed in SS; can use a delayed treatment control group - When the random assignment is not possible
When can’t get enough participants - useful for studying rare events - When wanting a lot of detail on participants, or intervention modifications, settings, etc.
- When expecting behaviours to change when conditions change (e.g., treatment introduced/withdrawn)
- When don’t have the resources to do group
- Useful for clinical innovation
How would you denote different stages of a study in Single-subject designs?
- Different letters denote different stages of a study
A = baseline, no treatment
B = first treatment
C = second treatment, different than B
B’ = first treatment with small variation - Use subscripts to denote repetitions of a segment (e.g., A1, A2, etc.)
- Observed behaviour (DV) is plotted on Y-axis with time on X-axis
Explain the baseline step:
- Repeated, continuous measurement of DV before begin any manipulation, such as a treatment
- Control Condition
- Assumed to represent how DV would behave without intervention
- Good Baseline
Multiple measures (minimum needed 3-4; ideally more)
Stable
Limited variability
No clear trend up or down
No ceiling/floor effects
Room for improvement/Opportunity for contrasting results
What are the types of baseline? (4)
Stable (IDEAL)
Variable
Stable accelerating/decelerating
Variable accelerating/decelerating
What are the length of the phases?
- Baseline
Until get stability
Minimum 3-4; more is better - Treatment(s)
Repeated, continuous measurement
Until get stability ideally
Minimum 3-4; more is better
Influenced by expected rate of change
Target
Type of client
Type of intervention
Can be set by length of time/number of sessions OR until achieve criterion
Frequency of measures is also influenced by expected rate of change
Explain Phase A:
Designs or Case Studies - not Single subject
They are not hypothesis testing
Just a ‘baseline’
No experimental manipulation, only a DV
No experimental control
Careful and systematic descriptions of 1 or a few interesting or unusual case(s)
Enhanced by attempts to quantify observations
Case series: description of a group of similar cases
Explain Phase B:
Observe effects of treatment over time - not SS
DV – behaviour of interest
IV – treatment
Unable to test causal relationships between the IV and the DV as no experimental control
Don’t know what DV is like without treatment, what the normal variability is
Factors other than treatment may have influenced change in DV seen during Phase B
? Threat to internal validity
What are A-B pre-experimental designs?
A = baseline
B = treatment
Not experimental as no experimental control
Unable to test causal relationships between the IV and the DV as no experimental control
Here you do know what DV looks like without treatment (i.e., A)
Factors other than treatment may have influenced the change in DV seen during Phase B
What are experimental designs?
- Experimental (single subject designs) are hypothesis testing
What are the two ways to establish experimental control in a variety of ways?
Replications
Control Goals
What are the benefits of increase control through replication?
- Increases internal validity
The more frequently an effect can be replicated in a design, the stronger control against threats to internal validity and the more able to attribute change in DV to intervention - Increases external validity – i.e., generalizability
Systematically plan studies to vary across conditions
Subjects, settings, personnel performing intervention, timing of intervention, equipment, etc. - Interpretation: Rx is effective when
Behaviour changes when Rx is implemented and does not change for remaining baselines
Behaviour changes only when the Rx is implemented and does so directly or closely after implementation
What are withdrawal designs?
- Replication of phases: Withdrawal designs
A1 B A2 design
Involves a second no treatment period (has 2nd baseline)
If the DV reduces in second baseline (A2), can more easily attribute change during B to treatment
What are withdrawal designs?
- Replication of phases: Withdrawal designs
A1 B A2 design
Involves a second no treatment period (has 2nd baseline)
If the DV reduces in second baseline (A2), can more easily attribute change during B to treatment - Additional control can be built into a withdrawal design by adding more phases
e.g., A1 B1 A2 B2 design
Provides 2 opportunities to evaluate the effect of treatment
If DV declines or levels off in A2 and then improves again in B2, demonstrates consistency of response
Therefore, strong support for effect of treatment
What would be problems with a 2nd baseline?
Problems with a 2nd baseline:
Contrary to behaviorist theory (the underpinnings of SS designs), improvement may be irreversible
Many treatments designed to develop independent learning – strategies individual can apply on their own
Withdrawing a treatment that is working is potentially unethical
? Possible way around?
? Study where it would be appropriate?
In Intervention Research, rather then ABA, generally viewed as ABC where C is maintenance
Assumes new skill remains when intervention is withdrawn
True goal of intervention
What are multiple baseline designs?
Replication across subjects or conditions (settings, behaviors, therapists…)
Concurrent monitoring
Establish stability in baselines
Introduce treatment to all at the same time
Non-concurrent monitoring
Arbitrarily define different baseline lengths
Randomly assign length of baseline to subjects or conditions
Alternative, introduce treatment to second subject after response to treatment in first becomes stabilized
? – which is preferable? Why?
Controls for maturation, history, other threats to internal validity
Explain Multiple Baseline Designs across behaviors:
- Two or more related yet functionaly independent behaviours (DVs) within a participant
- Vary length of baseline for each behaviour
Controls for external forces being responsible for change
Strengthened when non-targeted behaviours remain stable until they are targeted
where can you randomize?
Explain Multiple Baselines across Behaviours with control goal:
- Treatment goal
directly treat - Generalization goal
Related, expectation that these will be affected by training on treatment goal - Control goal
Unrelated, don’t expect to change
Controls for external factors (e.g., maturation)
Difficult to choose
?Where can you randomize?
Explain Multiple Baseline across subjects:
Single subject design repeated with similar participants
Looking to replicate findings
Replication increases internal validity
Increases external validity/generalizability somewhat
Non-concurrent (or staggered or variable) baseline important for internal validity
Explain Multiple Baseline across settings:
- Single subject design repeated with same individual(s) in different settings
- Typically used to show generalization – what you want in treatment
Settings can be somewhat like generalization goal – expect to see delay/lesser impact but may see change - With some individuals, may predict that you need to treat in each setting – use as experimental control (don’t see change until treat in each setting)
Explain alternating treatment designs:
- Theoretically, 2 treatments should have independent and differential effects
- Including a second treatment clarifies the impact of each on the DV
- A B C B C B etc. design: Compares the effects of rapidly alternating treatments B and C (where 1 of the treatments may be a placebo)
- Problem 1: Treatment reactivity
Must only compare adjacent phases
C cannot be assessed separately from B
Relationship between response to C and baseline (A) is unclear
A B A C: Introduces a 2nd baseline: alleviates reactivity somewhat
Order effects might be addressed using an A B C A C B design
Can be controlled through replication across subjects with alternate ordering - Problem 1: Treatment reactivity
How to reduce reactivity in alternating treatment designs?
- Sometimes treat one set of targets with B and another set with C to reduce reactivity
- Alternate design
Can see studies where both treatments done simultaneously
Used different items in each treatment - Need to question whether effective for reactivity based on treatment and target
Explain interaction designs:
- Purpose – to examine the impact of pieces of a complex intervention
- Reduction Design
Total package evaluated and then compare to single component
A-BC-B-BC-B - Additive Design
Determine effect of a simple protocol then add another component
A-B-BC-B-BC - Allow you to look at interactions among components of treatment
Order/reactivity effects are not eliminated
Can be controlled through replication across subjects
Explain Multiple Treatment designs:
- Interactive Designs
A B BC (simplest): Allows analysis of combined (BC) as well as separate (B) treatment effects
Variations on this theme: e.g., A BC B BC, etc…
Can potentially isolate impact of individual treatment components - Like Interaction Designs but view treatments as distinct rather than 2 parts of a package
What are 4 scientific methodology components?
Operational Specificity
Repeated Measures
Interobserver agreement
External validity
Explain operational specificity: (IV)
- IV (treatment)
Instructions to participant
Stimuli/materials
Response expected
Feedback/reinforcement
Scheduling & amount of treatment
Etc. - Essentially so could replicate
- Treatment fidelity evidence
Explain operational specificity: (DV)
- DV (outcome measure(s))
Trial scoring (% correct – need obligatory context)
Rate measures
Frequency measures
Interval and Time sampling
Must set criterion level of performance – when you’ll say goal was mastered
More likely to evaluate outcomes in multiple ways than in group designs - Participant(s) characteristics
- Interventionist characteristics
- Setting characteristics
How are repeated measures done? (3)
Done in a consistent way
At regular intervals
More frequently than in group designs
What are the 2 types of analysis of Single-Subject Designs?
Visual
Statistical
What is a threat to internal validity from repeated measures?
Testing effect (subject gets better with time)
What do you do in Visual Inspection when analyzing single-subject designs?
- Compare adjacent phases
Variability: within each phase
Level: value of dependent measure
Last point of previous phase to 1st point of following
Average of adjacent phases: misleading with high varibility or sharp slopes
Trend: direction of change in a phase
3 patterns: Accelerating, decelerating, flat
Slope: rate of change
What is a Celeration line?
Compare slopes
Compare overlap in data points
Best fit lines
