Lecture 17: Within-Subjects Designs Flashcards
within-subjects designs
Within-subjects experimental design uses a single group of participants in all conditions
synonyms of within-subjects designs
within-group, within-participant design, or repeated-measures design
what two things does a within-subjects design accomplish?
- Equating groups by using the same subjects
- Reducing within-group variance by controlling for individual differences
individual differences in within-subjects designs
- Individual differences are eliminated
- Controlling for individual differences increases sensitivity and thus the ability to detect a treatment effect
error variance in within-subjects deisgns
Error variance is reduced considerably because the participants become their control
F-ratio for within-subjects designs
F= condition effects + error/ error
variability in within-subjects designs
- Variability associated with individual differences is removed (it contributes equally to the numerator and denominator)
- There is no assumption of independence between condition scores as there is in a between-subjects design because each individual contributes to each condition
- Between-condition variance is based on within-subject comparisons
two sources of potential confounding in within-subjects designs
- confounding from environmental variables
- confounding from time-related variables
confounding from environmental variables
characteristics of the environment that may change across the range of conditions that each participant must complete
confounding from time-related variables
between the conditions, participants may be influenced by factors other than the treatments being investigated (fatigue, practice, etc.)
power of within-subjects designs
they reduce the within-group variance and gives a more powerful test
environmental variables
Any characteristic in the environment that may differ between treatment conditions
example of an environmental variable
noise, lighting, experimenter
impact of environmental variables
- they can become confounds
- we can no longer say the treatment caused the outcome
how can we control environmental variables?
- Standardizing
- Holding constant the environment across conditions
- Matching across treatment conditions
- Randomization
big 5 time-related factors
- history
- maturation
- instrumentation
- regression toward the mean
- testing effects
history
when an outside event changes over time and affects Ps scores in one condition but not the other
maturation
changes in Ps’ physical or psychological characteristics between treatments
instrumentation
changes in the measuring instrument throughout the study
regression toward the mean
extreme scores often move toward the mean on a second test
testing effects
when scores are affected by experience in prior condition (fatigue, learning, boredom)
order effects
directly related to the experience obtained in a research study
what time-related variables are related to the length of time between conditions?
history, maturation, and instrumentation
length of time between conditions and the impact of environmental variables
- If short timespan (1 hr) between conditions, less likely that these changes will occur
- If longer timespan (weeks or months), chances increase that time-related changes will influence results
how to reduce the effects of history, maturation, and instrumentation
- Decrease the time between conditions to reduce the likelihood of this happening
- Counterbalance: matching treatments with respect to time
order effects
- Effects that one treatment may have on another treatment
- Influenced by events or experiences that occurred earlier in the sequence of conditions
what designs are prone to order effects?
within-subjects designs
types of order effects
carryover and progressive effects
carryover effects
exposure to one manipulation that produces persistent consequences influencing the participants’ response to subsequent manipulations
progressive error
changes to behaviour/performance that are related to general experience in a research study (but not because of the treatment)
types of progressive error
practice and fatigue effects
practice effects
progressive improvement through treatment conditions
fatigue effects
progressive decline in performance through treatment conditions
problem with order effects
does the change in performance between conditions result from differences in the IV or order effects
solution for dealing with order effects
counterbalancing
time-related design challenges
- The possibility of a time-related threat (history, maturation, instrumentation) is directly related to the length of time required to complete the within-subject study.
- Increasing the time between treatments increases the risk of time-related threats to internal validity
- Reducing the time between treatments increases the likelihood that order effects will influence results.
- Between-subjects design may be a better choice for research conditions that are prone to order effects.
counterbalancing
Changing the order in which conditions are administered from 1 participant to the next so that they are matched overall
goal of counterbalancing
to use every possible order of treatments with an equal number of subjects participating in each sequence
purpose of counterbalancing
to eliminate time-related confounding
impact of counterbalancing
- Disrupts the systematic relationship between treatment order and any order effects
- Prevents order effects from accumulating in a particular treatment condition (spreads evenly)
complete counterbalancing
- All possible treatment orders are used equally often
- There are equal numbers of participants in each treatment condition
logical counterbalancing
- A particular series of treatment conditions may create their own unique order effect
- Therefore, include every possible ordering of treatment conditions
- Does not eliminate order effects, just controls them
requirement of counterbalancing
There must be equal numbers of participants in each counterbalanced order
issues with complete counterbalancing
- As the number of conditions increases, complete counterbalancing becomes more complex and # of required participants increases!
- Complete counterbalancing requires too many conditions (and subjects per condition)
latin square counterbalancing
- Each condition occurs equally often in each order in the experiment (ex, for 3 conditions: ABC, BCA, CAB)
- Each condition occurs exactly once in each order
- Equal numbers of participants are assigned to each order
- Instead of all 6 possible orders (3 x 2 x 1), the Latin Square requires only 3 orders
history of Latin square counterbalancing
- Developed from the agricultural rotation of crops across plots of land to avoid draining the soil of crop-specific nutrients
- Latin squares attributed to Euler (1750s) and Fisher (1935)
- Named after Euler’s use of Latin characters as a symbol
partial Latin square counterbalancing
- Each treatment condition occurs equally often in different sequence positions across the orders
- In partial counterbalancing, a Latin square can be constructed to decide which sequences to select
- In this counterbalancing, each condition is preceded and succeeded equally often by the same conditions
alternative method of partial Latin square counterbalancing
- Changes the condition order so that it is preceded and succeeded by different conditions
- In this counterbalancing, the Latin square is adjusted to balance the order of conditions that precede and succeed each condition
- In this counterbalancing, each condition is NOT preceded and succeeded by the same conditions
limits of counterbalancing
- Carryover effects can be asymmetrical
- Counterbalancing the Condition orders (A, B versus B, A) does not yield a similar carryover effect
- Asymmetries mean that the counterbalancing order can interact with the IV to influence the DV
- Range effects in within-subjects designs: participants may be influenced by the range of tasks they are given
example of asymmetrical carryover
- Pilots were tested on 2 ground steering methods for airplanes (manual): “Rudder pedal” and “Steering handle”
- Dependent variable: accuracy of steering
- 2 counterbalancing orders:
1) Rested, then Fatigued
2) Fatigued, then Rested - Results: Pilots performed worse overall in the counterbalanced order fatigued, then rested than in the order rested, the fatigued
- Pilots performed worse on the Rudder pedal in order = Fatigued first. They performed the same on the Rudder pedal and Steering handle in order = Rested first
- Possible explanation: Less learning (causing carryover) occurs during Fatigue and so… More carryover (learning) occurs in Order 1 (Rested, then Fatigued) than in Order 2 (Fatigued, then Rested)
- Counterbalancing the Condition orders (Fatigued / Rested versus Rested / Fatigued) did not yield similar carryover effects.
- Asymmetries mean that the counterbalancing order can interact
with the IV (type of brake) to influence the DV (performance).
example of range effects in a within-subjects design
- A manual dexterity test performed on a table that has a changing height
- Absolute table height = between-subjects variable (High or Low)
- Relative Table height = within-subjects variable
- High: Relative Table height was centred at 0 inches relative to their elbow
- Low: Relative Table height was centred at 6 inches height below their elbow
- Range effects in within-subject designs: Participants may be influenced by the range of tasks they are given
- Results: Peak performance for each group is influenced by the range of values they experience
- High Group performs best around -1 inch (near elbow height)
- Low Group performs best around 6 inches (below elbow height)
- Both groups were influenced by the within-subject IV (Relative table heights):
- But also influenced by between-subject IV (Absolute table height):
- High Group is best at 0 inches above the elbow
- Low Group is best at 6 inches below the elbow
- Implication: Range effects can be reduced by keeping as many variables constant as possible between subjects when using within-subject designs
reversibility
IVs that permanently alter the development or state of participants in irreversible carry-over effects
examples of reversibility
- Learning conditions; particular treatments to improve a skill or behaviour (cannot be “unlearned”)
- Physiological changes (brain lesions)
- Some medications or chemicals
when are within-subjects designs not appropriate?
Within-subjects designs are not appropriate if the experimental conditions produce a lasting effect on the participants that cannot be reversed
order of administration and reversibility
- Condition A = measure behaviour at baseline
- Condition B = measure during praising intervention
- Condition A’ = measure after intervention stopped
advantages of within-subjects designs
- Fewer participants are required (ex. 3 conditions with 30 participants):
- Eliminates problems of individual differences
- Can increase the chances of detecting a treatment effect
disadvantages of within-subjects designs
- Not suitable when carryover effects are expected
- Participant attrition may be a problem
- Ordering of conditions can be time-consuming and require many participants
does counterbalancing eliminate order effects?
- Counterbalancing does not eliminate order effects
- Adds the order effects to some (but not all) of the subjects within each treatment
comparing designs at the analysis stage
Different analyses (within- or between subjects) can yield similar results but the variability across stimuli can differ from the variability across participants
major weakness of between-subjects designs
individual differences
three factors that differentiate between- and within-subjects designs
- Individual differences
- Time-related factors and order effects
- Number of required participants
what study design should you choose?
Choose the study design by the factors of most interest to the study to avoid problems of validity
ABA’ design
Allows one to measure the presence or absence of carry-over effects
example of an ABA’ design
Using praise with children in the classroom to increase participation
major weakness of within-subjects designs
order effects
major strength of between-subjects designs
eliminating order effects
major strength of within-subjects designs
eliminates variability from individual differences; needs fewer participants
