Weeks 3-4 Flashcards
what are three types of between-subjects design?
a) Post-test only
b) Pre-test/post-test
c) Matched pairs
In a post-test design, the IV manipulation doesn’t have to be…
The IV manipulation doesn’t have to be something you do to participants (i.e., treatment). It can also be something about the experiment which is manipulated (i.e., time).
i.e., the race of the defendant
in a jury decision-making
study (IV) and the length of
the sentence
recommended is the (DV).
what is a post-test design?
A between-subjects research design in which the same assessment measures are given to participants both before and after they have received a treatment or been exposed to a condition, with such measures used to determine if there are any changes that could be attributed to the treatment or condition.
what is a matched pairs design?
A matched pairs design is a between-subjects experimental design that is used when an experiment only has two treatment conditions. The subjects in the experiment are grouped together into pairs based on some variable they “match” on, such as age or gender. Then, within each pair, subjects are randomly assigned to different treatments.
what is a pre/post-test design?
A between-subjects research design in which the same assessment measures are given to participants both before and after they have received a treatment or been exposed to a condition, with such measures used to determine if there are any changes that could be attributed to the treatment or condition.
what are the advantages and disadvantages of a post-test-only design?
Advantages
Skipping the pre-test makes it harder for participants to guess the hypothesis and reduce likelihood of demand characteristics (threat to internal validity).
Efficient: faster (time) and economical (cost) effective makes it less likely for participant drop-out/attrition (mortality effect).
Disadvantages:
No pre-test to provide a baseline to compare change to determine how much the treatment (IV) changed the DV (i.e., within-subject comparison).
We have no demographic information or anxiety scores to ensure that random assignment was effective in creating equivalent groups (must take it on faith that random assignment was effective in controlling for extraneous variables).
Can’t evaluate mortality effects (no way to compare drop-out and completers to ensure they’re not systematically different).
what are the advantages and disadvantages of a pre/post-test design?
Advantages:
We can determine if random assignment worked, and the groups are equivalent before we start the study and can use a smaller sample size (if we want).
We can look at change in the DV rather than anxiety at one point in time (amount of change in an individual; not just between groups comparison).
We have a baseline to compare drop-outs and completers with to assess mortality rates and can therefore correct for them if they’re present.
More control over extraneous variables (pre-existing anxiety in this case) and focusing on how much change in the DV occurs across each condition means that it doesn’t matter that the groups weren’t equivalent on pre-existing anxiety.
Disadvantages:
More time consuming and costly (time and money)
Makes it easier to guess the hypothesis (demand characteristics)
Doesn’t suit every type of research question.
May increase attrition/drop-out rates.
Is vulnerable to history effects (solution remove pre-test or have the control group over the same time period as the control).
Pre-test can contaminate the post-test data in some way (exposure, practice, carry-over = testing effects).
equivalent scores at pre-tests indicate…
that random assignment was successful.
does having equivalent groups matter in pre/post-test design?
No. we are focusing on the change in the DV, not a score at one point in time.
what are the advantages and disadvantages of a matched pair design?
Advantages
More experimental control by matching on extraneous variables rather than leaving it to chance (reduce noise in data; competing variability to IV).
Fewer participants are needed to achieve equivalent groups (i.e., expensive study or rare participants are good for this design).
Disadvantages Testing effects (same as pre-test/post-effects)
Time and money (can be hard to find participants that match).
Difficult to find match (especially if you want to match people on multiple variables).
But smaller samples means the random assignment is less effective in creating equivalent groups on the other non-matched extraneous variables.
How do you deal with dropouts; drop both people in the match is better but sad to due, there are statistical work arounds you can use to get around this.
what is the process of a matched pairs design and what can we match for?
It has more __ than random assignment alone.
Matched Pairs Design
(Use pre-test to match people on subjective factors to make groups equivalent)
i.e., take the two most anxious people, flip a coin and assign them to a condition, then the next two and so forth
This will ensure that the groups are equivalent in pre-existing levels of anxiety (DV). The experimenter is taking control of the extraneous variable and not leaving to chance that they will be equally distributed across groups (the levels of it). You can match people on ANY important extraneous variable (i.e., age, gender, ethnicities, education, SES etc.). Is good when the sample size is small to ensure that the groups are equivalent and not leaving it to chance with random assignment. Doesn’t have to be the pre-test variable it can be any extraneous variable!
Matching has better control than just using random assignment!
How does a within-subjects design control help with matching for extraneous variables?
The problem with this design? What is the confound?
What is a solution?
How do we calculate the number of
orders?
What do we use to determine their order?
A second solution?
How does a within-subjects design control help with matching for extraneous variables?
- All the variables are matched because the
same participants are in every condition
meaning their extraneous variables, they
bring with them will be equivalent as well. - This is the best way to control for
extraneous variables.
The problem with this design? What is the confound?
- testing effects (i.e., fatigue or practice
effects) - where performance in one block impacts
later performance on another block. - People get better with practice, or they
perform worse due to bored or fatigue from
repetitive trials.
Solution is to counterbalance the order of
blocks.
- If two conditions ½ do AB and the other ½
do BA order.
- It doesn’t remove fatigue or practice
effects, but it ensures that if they are
present, they effect each trail equally.
- If three conditions, there are six possible
orders. ABC, ACB, BCA, BAC, CAB, CBA.
How do we calculate the number of
orders?
- 3! (3x2x1) if 4! (4x3x2x1 = 24).
This can get quite complicated the more
conditions you have to solve this people
use the Latin square (to ensure that each
condition have equal likelihood of being in
the first or last position where testing
effects will be greater).
- Subject 1 = ABCD, Subject 2 = BCDA,
Subject 3 = CDAB, Subject 4 = DABC.
- What have we lost from this method? B
always come after A and not C or D (small
order effect remains).
If we can, randomly intermix the trails in the
blocks?
- Would need to change the instructions to
search for the emotional face
- Stimuli are equally distributed across
conditions (blocks) meaning practice and
fatigue effects influence them all equally.
Within-Subject Designs
Advantages:
Disadvantages:
Advantages:
- Best control for extraneous variables
- Smaller sample sizes are needed
(extraneous variables are matched and
to fill conditions)
- More statistical power
Disadvantages:
- Order effects
- Asymmetric transfer (can not be fixed
with counterbalancing; easy vs hard
tasks; going from easy to hard will need
more practice to be as good as those
who went from hard to easy; hard-easy
more practice; inequivalent conditions;
for example, from neutral to fearful vs
fearful to neutral have fear responses
that haven’t gone away yet).
- Contrast effects (contrast between
conditions may lead to participant
guessing the hypothesis from the IV;
social desirability of not wanting to be
perceived as racist; IV becomes salient).
- More time for a session (fatigue leads to
poor data)
(4) Factorial Designs
(4) Why add another variable[s]
2 or more independent variables
Each variable can be manipulated
within- or between-subjects
Each variable can have 2 or more
levels
Some variables can be subject variables
(quasi-experiment; can’t manipulate
subject variables).
- Save time. Assess 2 or more potential
causes at once (relative to multiple
studies) - To refine a theory (because it depends…
to identify what situational factors the
effect of the IV depends on). - To rule out confounds.
- To increase external validity (extend to
other populations, stimuli, situations…;
good for quasi-experimental designs).
Summary:
5 experimental designs
You can have multiple …. in a study
Experimental Designs
- Post-test only
- Pre-test/Post-test
- Matched Pairs
- Within-Subjects
- Factorial Designs
Independent variables
• Manipulated
• One or more IVs
• Each with 2 or more levels
• Manipulated between- or within-
subjects
• Usually categorical, but might reflect an
underlying continuous variable
Dependent variables
• Measured
• Can be categorical or continuous
• Can include more than 1 DV in a study
(3) Assumptions of an Independent Student’s t-test
1. Samples are independent (observation
in Group 1 are independent of
observations in Group 2.
2. Dependent variable is normally
distributed.
3. The variance in the two groups is
approximately the same (SD)We want our assumption checks to be…
We want the assumptions to be NON-SIGNIFICANT. If they’re significant we are rejecting the null hypothesis that these groups are the same. Therefore, they’re not sampled from the sample continuous distribution.
Independent t-test: If we violate homogeneity of variance….
Welch’s t-test: if the variance is not equal across groups (heterogenous) than we should see that the DF will be significantly smaller than the student’s t DF!
DF will be smaller and therefore is more conservative because it is less likely to produce a false significant effect (it penalizes you for not having clean data by squashing the t distribution and increases how big the t needs to be to be significant at p-value .05).
Independent t-test: Violations of Normality….
If we violate normality (Shapiro-wilk test)
• Data is not normally distributed,
common in small samples.
• If significant than we do not have a
normally distributed data set.
• Solution: a less strict t-test. The Mann
Whitney U (non-parametric test
Mann-Whitney U Test
1. Non-parametric test
2. Rank order all the participant RTs
(fastest to slowest; label which group
they’re in)
3. Null hypothesis – all RTs are equally
likely to come from either condition
4. Research hypothesis – the faster RTs
are more likely to come from the angry
condition, and the slower RTs are more
likely to come from the happy
condition.
5. Use for non-normal data, especially if N
is small.
Logic:
If the angry group is really faster than happy faces, then the data should be ordered where fastest times are all angry and slowest times were all the happy conditions.
If the null hypothesis is true the labels will be mixed because participants fats, medium and slow would be equally as likely to be in angry or happy condition. If we take the sample and split it in two, if most angry fall one way and all the happy fall the other way this is unlikely. How unlikely is it? That’s your p-value.
It doesn’t care how big the difference is (numbers) all it cares about is how many people in the angry condition are at the fast end and how many in the happy condition are at the slow end (as a group; is the angry group faster than happy group).
Degrees of Freedom:
within
between
chi square
Degrees of Freedom:
DF: N-1 (Within-subjects)
DF: N-2 (Between-subjects)
DF: (Row-1) x (Column-1) – chi square
t-test statistics:
independent vs paired
within:
difference between conditions/standard error of the difference
between:
mean group difference/SE
Paired Samples T-Test is also called:
Also called
o Dependent t-test
o Matched t-test
o Repeated measures t-test
Assumptions of paired-samples t-test
Assumptions (paired t-test)
1. Pairs of observations are independent
(each person in sample is independent
of one another; provide a pair of
scores)
2. DV (difference score) is normally
distributed.
Paired samples t-test: If we violate normality…
Wilcoxon Signed Rank Test (non-parametric version of a paired t-test)
It ranks observations in order from
fastest to slowest, if data is grouped
into x2 conditions (angry-fast; happy-
slow) than null hypothesis is rejected
because the findings are not likely to be
due to chance.
More conservative to paired t-test.
Chi Square is used if my DV is …
*What if my DV is categorical?
Do unrealistic beauty standards in the media contribute to eating disorders? DV: which snack will you choose rather than a survey (nominal or categorical data).
We use a chi square test to test effect of the IV manipulation (fashion or home magazine).
In contingency table: if the null hypothesis is true, we would expect the proportions to be equal across conditions. In this example, those who read the fashion magazine were more likely to choose apples than chocolate and those who read the home decorating magazine were much more likely to choose chocolate. Pattern present. Reject null hypothesis.
