research methods exam 3 Flashcards
3 criteria for a causal claim
-temporal precedence
-covariance
-no extraneous variables
construct validity
-how well a conceptual variable is
operationalized/defined
-When you ask how well a study measured or manipulated a
variable, you are interrogating the construct validity
effect size (r)
the strength of a relationship between two or more variables
-indicates the importance of a relationship (weak, moderate, strong)
confidence intervals
precision of results
-results of statistically significant when the CI doesn’t contain zero
-given range indicated by a lower and upper value that is designed to capture the population value for some point estimate (e.g., percentage, difference, or correlation)
- a high proportion of CIs will capture the true
population value
-more people shrinks the CI, less people makes CI smaller
-if not given p value in results, use CI: significant results will not contain 0
third variable problem
it is possible for a third variable (confounding/extraneous) to be causing spurious correlations
-we shouldn’t underestimate this possibility of these types of unmeasured variables
independent variable
the manipulated variable
-assigning participants to be at one level or the other
dependent variable
the measured variable
-aka the outcome variable
control variable
variable that experimenter holds constant on purpose
counterbalancing
-a way to avoid order effects
-presenting the levels of the IV to participants in different sequences
-this should cause order effects to cancel each other out when all of the data is combined
-counterbalancing can be “full” or “partial”
-full: all possible condition orders are represented
-partial: some, but not all, of the possible condition orders are represented
manipulation check
an extra dependent variable that researchers can insert into an experiment to convince them that their experimental manipulation worked
within-subjects design
an experimental design in which each participant is presented with all levels of the independent variable
between-subjects design
-aka independent-groups design
-experimental design in which different groups of participants are exposed to different levels of the IV, such that each participant experiences only one level of the IV
matched design/group
an experimental design technique in which participants who are similar on some measured variable are grouped into sets
-the members of each matched set are then randomly assigned to different experimental conditions
pre-post design
experiment using an independent-groups design (between-subjects) in which participants are tested on the key dependent variable twice: once before and once after exposure to the IV
repeated measures design
an experiment using a w/i groups design in which participants respond to a dependent variable more than once, after exposure to each level of the IV
post-only design
experiment using an independent-groups design (between-subjects) in which participants are tested on the DV only once
concurrent measures design
an experiment using a w/i-groups design in which participants are exposed to all the levels of an IV at roughly the same time, and a single attitudinal or behavioral preference is the dependent variable
order effects
one of 12 threats to internal validity
-in a w/i-groups design, threat where exposure to one condition changes participants responses to a later condition
selection effects
one of 12 threats to internal validity
-when the kinds of participants in one level of the IV are systematically different from those in the other
-can happen if participants choose which group they want to be in OR if researchers assign one type of person to one condition (ex:women) and another type of person to another condition (ex:men)
design confounds
one of 12 threats to internal validity
-experimenter’s mistake in designing the IV
-occurs when a second variable happens to vary systematically
along with the intended independent variable
-the accidental second variable
is therefore an alternative explanation for the results
factorial notation/design
-a way for researchers to test for interactions
-design is one in which there are two or more IVs (also referred to as factors)
-most common factorial design:
researchers cross the two independent variables; study each possible combination of the independent variables
main effects
in factorial designs, the overall effect of one IV on the DV, averaging over the levels of the other IV
-the number of IVs = number of main effects
interaction effects
-result from a factorial design, in which the difference in the levels of
one independent variable changes, depending on the level of the other
independent variable
-a difference in differences
-aka an interaction
within, between and mixed designs
between-groups factorial design:
-both IVs are studied as independent-groups
-if the design is a 2×2, there are four different groups of participants in the experiment
within-groups (or repeated-measures) factorial design:
-both independent variables are manipulated as within-groups
-If the design is 2×2, there is only 1 group of participants but they participate in all 4 combinations of the design
mixed factorial design:
-one independent variable is manipulated as
independent-groups and the other is manipulated as within-groups
interaction graphs
in interaction graphs where the variables DO interact, there will be lines crossing
-interaction graphs can look different, but if there is interaction, then the lines will be crossing at some point
-no interaction = no crossover
history threat
one of 12 threats to internal validity
-a “historical” or external factor that systematically affects most members of the treatment group at the same time as the treatment itself
-makes it unclear whether the change is caused by the treatment received
-external factor must affect most people in the group in the same direction (systematically), not just a few people (unsystematically)
ex: why did the dorm residents use less electricity?
Was it the Go Green campaign? Perhaps. But a plausible alternative
explanation is that the weather got cooler and most residents did not use air conditioning as much
-to fix this: use comparison groups
attrition threat
one of 12 threats to internal validity
-when only a certain kind of participant drops out of the study before it ends
-a reduction in number of participants from pretest to posttest
to fix this: look at the dropout levels (from which groups?) to see if the study groups remain equal
maturation threat
one of 12 threats to internal validity
-change in behavior of the experimental group that occurs spontaneously over time
-to fix this: use comparison groups
instrumentation threat
one of 12 threats to internal validity
-when a measuring tool changes over time
-makes it unclear if change is from different test or actual change
-to fix: mirror pre/post, use post-only and have a clear coding guide
regression threat
one of 12 threats to internal validity
-extremely low or high performance at time 1 is likely to be less extreme at time 2 (closer to the average)
-happens only when measuring twice & with extreme scores
-to fix: use comparison groups
testing threat
one of 12 threats to internal validity
-type of order effect in which participants’ performance improves over time because they become practiced at the dependent measure
-aka fatigue effect or practice effect
observer bias
one of 12 threats to internal validity
-bias caused by researchers expectations influencing how they interpret the results
-to fix: use a double-blind study
demand characteristics
one of 12 threats to internal validity
-a cue that leads participants to guess a study’s hypothesis or goals
-participants change their behavior in the expected direction
to fix: use a double-blind study, masked as best as possible
placebo effects
one of 12 threats to internal validity
-people receive a treatment and improve, but only because they believe they are receiving a valid and effective treatment
to fix: add a 3rd group that receives nothing (not even placebo pill)
when to use each type of test (t-test, ANOVA, etc.)
independent-samples T test:
-comparing b/w subjects, 2 different groups
T test:
-analyzing simple experiments
repeated-measures T test (aka paired samples T test):
-comparing w/i subjects (participants see both conditions), 2 means/groups
-using pre-post test
1-way ANOVA:
-3+ groups
Bivariate correlation
-2 measured, ordinal, or ratio variables
Pearson correlation
-finding relation between quantitative variables
possible reasons for not enough between-groups difference
-floor and ceiling effects (floor: all scores clustered at low end, ceiling: all scores clustered at high end)
-weak manipulation of variables (ex: difference in violence after 1 hr or 2 hrs on video games - not enough difference)
-insensitive instruments (researchers have not used an operationalization of the dependent variable with enough sensitivity; ex: If medication reduces fever by 1/10th degree, you wouldn’t be able to detect it with a thermometer that was calibrated in 1 degree increments)
bar graphs
comparing 2 means
-used for independent samples T test
-when you have 1 categorical and 1 quantitative variable
scatter plot
-when you have 2 quantitative variables
