Lecture 16: Between-Subjects Designs Flashcards
between-subjects designs
- A different group of participants is assigned to each condition
- Each group receives a different experimental treatment (value of the IV) and the groups are compared
key element of between-subjects designs
separate groups of participants are used for the different conditions
what differences are compared in a between-subjects design?
Participants’ data (on the DV) is compared across groups to look for differences
how many groups are there in a between-subjects design?
You can have any number of groups; it depends on the number of levels of the IV
how many scores on the DV are there per participant in a between-subjects design?
Because participants experience only one level of the IV, there is 1 score on the DV per participant
synonym for between-subjects design
independent-measures experimental design
measuring IVs in between- vs. within-subjects designs
- Some independent variables can only be measured in a between-subjects design (ex. age, gender)
- Other independent variables can be measured in a between-subjects or within-subjects design (ex. teaching method, video condition)
example between-subjects study
- 15-month-old infants watch an adult persist for 30 s in opening objects in an effort or no effort condition
- Control condition: the child sees the box but there’s no adult present
- This was done as a between-groups design with 34 infants/condition
- Infants who saw the adult persist in the effort condition show more attempts to open their box
- They used a between-groups design because infants have a limited attention span
how do you determine an effect of the IV in a between-subjects design?
by comparing the mean scores (DV) for each group
systematic variance
the difference in the DV between groups
non-systematic variance
the difference in the DV within groups
what type of variance do we want to minimize?
Non-systematic variance beacuse it’s an important source of error
what sources of variability do we use to calculate our test statistic?
systematic & non-systematic variance
2 components of between-subjects (systematic) variance
- Treatment effects
- Effects of chance factors (experimental error)
If we treated the groups the same way, would you expect to see the same scores on your DV?
No, it is impossible to perfectly match groups, therefore there are always some errors and some differences between means
experimental error
all chance factors not controlled for
types of experimental error for between-subjects designs
individual differences & variations in the testing environment
within-group (non-systematic) variance
- Any differences between subjects who are treated alike
- Within any given treatment group, all subjects have been treated identically and should have the same value for the DV
- Any variability can only be a result of chance factors
- Non-systematic (within-group) variance = experimental error
how do we form the F-ratio?
We use between-group and within-group variance, to form the condition or treatment index
effect of treatment effects on the F-ratio
F-ratio is responsive to the absence or presence of treatment effects (i.e. the effect of the IV)
treatment index formula
treatment index= between-groups variance/ within-group variance
treatment index modified formula
treatment effects + experimental error/ experimental error
how do you determine statistical significance in a between-subjects design?
you compare the between-group variance to the within-group variance
impact of large variances
- Large between-group variance is good
- Large within-group variance is bad
interpreting F-ratios
- Between-group variance > within-group variance = F-ratio is positive and large
- Between-group variance < within-group variance= F-ratio is near 0 and small
standard error bars
the measure of the variance in the data within each group
what type of variance will yield a non-significant F-ratio?
Small between-group variance or large within-group variance
comparing more than 2 groups in a between-subjects design
- You can use a single-factor multiple-group design
- Ex. comparing driving performance under three conditions: cell phone, handsfree phone, no phone
- Sometimes, using more than 2 groups provides stronger evidence for a real cause-and-effect relationship than a two-group design
what differences do researchers want to minimize and maximize
- Researchers try to maximize between-group differences (i.e. make sure you are comparing two distinct things)
- Researchers try to minimize within-group differences
how do experimenters minimize within-group differences?
- Make sure all participants within a group are treated the same
- Standardize the experimental procedures
- Minimize individual differences
- Hold extraneous variable constant or restrict the range
- Use a large sample size
effect of random assignment on variance
random assignment does not affect within-group variance
two major sources of confounding variables in between-subjects designs
- Individual differences (from assignment bias)
- Environmental variables
individual differences
any personal characteristic that differs from one participant to another
do we want treatment groups to be similar or different?
We want to make sure that the different groups are as similar as possible, except for the IV
assignment bias
when the process of assigning participants produces groups with different characteristics
assignment bias is a threat to ____
internal validity
environmental variables
any characteristics in the environment that may differ
impact of environmental variables on the validity of the study
- If these differ between groups, we may have an extraneous variable that becomes a confound
- We can no longer say that the treatment (IV) caused the outcome
- It could be an environmental factor
To establish equivalent groups of participants, a researcher must:
- Created equally
- Treated equally
- Composed of equivalent individuals
ways to limit individual differences and environmental variables
- randomization
- matching
randomization
- Participants are randomly assigned to groups to ensure groups are as equal as possible before treatment
- It is the most powerful technique to control for the effect of pre-existing differences by equalizing them (spreading them evenly)
- Randomly assigning individuals to different conditions to ensure even distribution of individual differences across conditions
randomization vs. random sampling
- Randomization is not the same thing as random sampling
- Random sampling: random selection of participants from a larger population to participate in the study
- Randomization: random assignment of participants to experimental or control groups in a particular study
free random assignment
- Coin toss to ensure that participants are assigned to groups solely based on chance
- Each P has an equal chance of being assigned to any one of the treatment conditions
- Theoretically, it should lead to equality, but there is no guarantee
importance of modifying randomization
- It’s improbable that groups will be perfectly matched, but will differ only randomly, which is typically very small
- Therefore, differences among groups are treatment effects and will neutralize nuisance effects at the same time
- With small samples, there are no guarantees
- Therefore, we can modify randomization to have better control over the outcomes
matching
- Participants are matched on critical variables
- This guarantees groups are equivalent on critical variables
4 steps of matching
- Identify the variable(s) to be matched and identify the potential confounding variables associated with this variable
- Measure and rank subjects on the variable for which control is desired (this may require a pretest)
- Segregate subjects into matched pairs on that variable
- Randomly assign pair members to the conditions
matching across blocks
- Matching can be extended to units larger than pairs
- Groups of individuals are matched in blocks
- Random assignment to groups from each block
threats to internal validity in between-subjects design
- attrition
- communication between groups
- resentful demoralization
attrition
Refers to participants leaving the study before completion
when is attrition a problem?
- Not a problem if members of all groups leave at the same rate
- This is a problem if they leave one group at a higher rate than in other groups (differential attrition)
- The groups are no longer equivalent
- Is the difference between groups due to treatment effects or differential attrition?
communication between groups
- Diffusion may occur
- Treatment effects spread from one condition to another condition
- The true effects of treatments may be masked by shared information
- One group is getting the benefit of information from another group
- There appears to be no effect
- Communication between groups can also result in resentful demoralization
resentful demoralization
- One group might receive course credit while another group might receive payment
- Any perceived inequity can influence behaviour
- In these cases, observed differences between groups have alternate explanations
advantages of between-subjects designs
- Simple design: each score is independent from other scores
- Clean and uncontaminated by other treatment factors like carryover and practice effects
- Takes less time for each participant
- Causality can be established
why are between-subjects designs popular?
- Because carryover effects are of unknown duration
- One can estimate the bias introduced by carryover effects
- Implication: carryover effects create more bias as the number of conditions increases
disadvantages of between-subjects designs
- Requires many participants
- It can be difficult to recruit enough people from special populations
- Individual differences and environmental differences can exist
- Generalization (external validity) can be hard if holding subjects constant on extraneous variables reduces their representation in the population
- Assignment bias, experimenter-expectancy and subject-expectancy biases
solutions to assignment bias, experimenter-expectancy, and subject-expectancy biases
Assign participants to conditions so that:
1) Participants are blind to (unknowledgeable about) the condition
2) Experimenters are blind to the condition
3) Both are blind (double-blind)
4) Analyst is blind to the condition of the participants
when to use a between-subjects design
To avoid comparison or carryover effects
when not to use a between-subjects design
When similar anchoring is desired across conditions
anchors
the interpretation of the labels at the endpoints of a rating scale
important considerations when choosing a between-subjects design
does the between-subjects design avoid:
1) Carry-over effects: the influence of one condition on other conditions
2) Participants’ awareness: sensitized to task measures over time
3) Ecological validity: if the participant is not usually exposed to all levels of a variable in nature, then the results are not representative
4) Changes in measurement properties/ tests over time
