Weeks 3-4 Flashcards
what are three types of between-subjects design?
a) Post-test only
b) Pre-test/post-test
c) Matched pairs
In a post-test design, the IV manipulation doesn’t have to be…
The IV manipulation doesn’t have to be something you do to participants (i.e., treatment). It can also be something about the experiment which is manipulated (i.e., time).
i.e., the race of the defendant
in a jury decision-making
study (IV) and the length of
the sentence
recommended is the (DV).
what is a post-test design?
A between-subjects research design in which the same assessment measures are given to participants both before and after they have received a treatment or been exposed to a condition, with such measures used to determine if there are any changes that could be attributed to the treatment or condition.
what is a matched pairs design?
A matched pairs design is a between-subjects experimental design that is used when an experiment only has two treatment conditions. The subjects in the experiment are grouped together into pairs based on some variable they “match” on, such as age or gender. Then, within each pair, subjects are randomly assigned to different treatments.
what is a pre/post-test design?
A between-subjects research design in which the same assessment measures are given to participants both before and after they have received a treatment or been exposed to a condition, with such measures used to determine if there are any changes that could be attributed to the treatment or condition.
what are the advantages and disadvantages of a post-test-only design?
Advantages
Skipping the pre-test makes it harder for participants to guess the hypothesis and reduce likelihood of demand characteristics (threat to internal validity).
Efficient: faster (time) and economical (cost) effective makes it less likely for participant drop-out/attrition (mortality effect).
Disadvantages:
No pre-test to provide a baseline to compare change to determine how much the treatment (IV) changed the DV (i.e., within-subject comparison).
We have no demographic information or anxiety scores to ensure that random assignment was effective in creating equivalent groups (must take it on faith that random assignment was effective in controlling for extraneous variables).
Can’t evaluate mortality effects (no way to compare drop-out and completers to ensure they’re not systematically different).
what are the advantages and disadvantages of a pre/post-test design?
Advantages:
We can determine if random assignment worked, and the groups are equivalent before we start the study and can use a smaller sample size (if we want).
We can look at change in the DV rather than anxiety at one point in time (amount of change in an individual; not just between groups comparison).
We have a baseline to compare drop-outs and completers with to assess mortality rates and can therefore correct for them if they’re present.
More control over extraneous variables (pre-existing anxiety in this case) and focusing on how much change in the DV occurs across each condition means that it doesn’t matter that the groups weren’t equivalent on pre-existing anxiety.
Disadvantages:
More time consuming and costly (time and money)
Makes it easier to guess the hypothesis (demand characteristics)
Doesn’t suit every type of research question.
May increase attrition/drop-out rates.
Is vulnerable to history effects (solution remove pre-test or have the control group over the same time period as the control).
Pre-test can contaminate the post-test data in some way (exposure, practice, carry-over = testing effects).
equivalent scores at pre-tests indicate…
that random assignment was successful.
does having equivalent groups matter in pre/post-test design?
No. we are focusing on the change in the DV, not a score at one point in time.
what are the advantages and disadvantages of a matched pair design?
Advantages
More experimental control by matching on extraneous variables rather than leaving it to chance (reduce noise in data; competing variability to IV).
Fewer participants are needed to achieve equivalent groups (i.e., expensive study or rare participants are good for this design).
Disadvantages Testing effects (same as pre-test/post-effects)
Time and money (can be hard to find participants that match).
Difficult to find match (especially if you want to match people on multiple variables).
But smaller samples means the random assignment is less effective in creating equivalent groups on the other non-matched extraneous variables.
How do you deal with dropouts; drop both people in the match is better but sad to due, there are statistical work arounds you can use to get around this.
what is the process of a matched pairs design and what can we match for?
It has more __ than random assignment alone.
Matched Pairs Design
(Use pre-test to match people on subjective factors to make groups equivalent)
i.e., take the two most anxious people, flip a coin and assign them to a condition, then the next two and so forth
This will ensure that the groups are equivalent in pre-existing levels of anxiety (DV). The experimenter is taking control of the extraneous variable and not leaving to chance that they will be equally distributed across groups (the levels of it). You can match people on ANY important extraneous variable (i.e., age, gender, ethnicities, education, SES etc.). Is good when the sample size is small to ensure that the groups are equivalent and not leaving it to chance with random assignment. Doesn’t have to be the pre-test variable it can be any extraneous variable!
Matching has better control than just using random assignment!
How does a within-subjects design control help with matching for extraneous variables?
The problem with this design? What is the confound?
What is a solution?
How do we calculate the number of
orders?
What do we use to determine their order?
A second solution?
How does a within-subjects design control help with matching for extraneous variables?
- All the variables are matched because the
same participants are in every condition
meaning their extraneous variables, they
bring with them will be equivalent as well. - This is the best way to control for
extraneous variables.
The problem with this design? What is the confound?
- testing effects (i.e., fatigue or practice
effects) - where performance in one block impacts
later performance on another block. - People get better with practice, or they
perform worse due to bored or fatigue from
repetitive trials.
Solution is to counterbalance the order of
blocks.
- If two conditions ½ do AB and the other ½
do BA order.
- It doesn’t remove fatigue or practice
effects, but it ensures that if they are
present, they effect each trail equally.
- If three conditions, there are six possible
orders. ABC, ACB, BCA, BAC, CAB, CBA.
How do we calculate the number of
orders?
- 3! (3x2x1) if 4! (4x3x2x1 = 24).
This can get quite complicated the more
conditions you have to solve this people
use the Latin square (to ensure that each
condition have equal likelihood of being in
the first or last position where testing
effects will be greater).
- Subject 1 = ABCD, Subject 2 = BCDA,
Subject 3 = CDAB, Subject 4 = DABC.
- What have we lost from this method? B
always come after A and not C or D (small
order effect remains).
If we can, randomly intermix the trails in the
blocks?
- Would need to change the instructions to
search for the emotional face
- Stimuli are equally distributed across
conditions (blocks) meaning practice and
fatigue effects influence them all equally.
Within-Subject Designs
Advantages:
Disadvantages:
Advantages:
- Best control for extraneous variables
- Smaller sample sizes are needed
(extraneous variables are matched and
to fill conditions)
- More statistical power
Disadvantages:
- Order effects
- Asymmetric transfer (can not be fixed
with counterbalancing; easy vs hard
tasks; going from easy to hard will need
more practice to be as good as those
who went from hard to easy; hard-easy
more practice; inequivalent conditions;
for example, from neutral to fearful vs
fearful to neutral have fear responses
that haven’t gone away yet).
- Contrast effects (contrast between
conditions may lead to participant
guessing the hypothesis from the IV;
social desirability of not wanting to be
perceived as racist; IV becomes salient).
- More time for a session (fatigue leads to
poor data)
(4) Factorial Designs
(4) Why add another variable[s]
2 or more independent variables
Each variable can be manipulated
within- or between-subjects
Each variable can have 2 or more
levels
Some variables can be subject variables
(quasi-experiment; can’t manipulate
subject variables).
- Save time. Assess 2 or more potential
causes at once (relative to multiple
studies) - To refine a theory (because it depends…
to identify what situational factors the
effect of the IV depends on). - To rule out confounds.
- To increase external validity (extend to
other populations, stimuli, situations…;
good for quasi-experimental designs).
Summary:
5 experimental designs
You can have multiple …. in a study
Experimental Designs
- Post-test only
- Pre-test/Post-test
- Matched Pairs
- Within-Subjects
- Factorial Designs
Independent variables
• Manipulated
• One or more IVs
• Each with 2 or more levels
• Manipulated between- or within-
subjects
• Usually categorical, but might reflect an
underlying continuous variable
Dependent variables
• Measured
• Can be categorical or continuous
• Can include more than 1 DV in a study
