4: Experimental research Flashcards
Experimental research - goal:
establish causal relationships
Types of research Qs:
- descriptive; eg. polls are just descriptive
- relational > correlational relationships
- causal > causal relationships
spurious relationships =
correlation confused with causation
3 necessary but not sufficient conditions to infer causality b/w X & Y:
+ problems
+ difficult point in research design
>>> overarching epistemologic problem
- co-variation = X & Y happen together
- time order: X before Y
- exclude alternative possible causes
+ reverse causation: Y actually causes X, or Z causes both
+ isolate potential cause from other factors
>>> causality can be inferred, but never proven 100%
3 steps to evaluate a treatment with an experiment:
+ 4 entities involved
ma.me.co:
- manipulate one or more IV = Independent Variables, eg who gets the treatment (in most basic design) in your TU Test Unit = population sample being tested
- measure the effect on the DV = Dependent Variable(s)
- control for the effect of EV = Extraneous Variables
+ IV, DV, EV & TU
Taxonomy of experimental designs
Experimental designs
- Quasi-experiment w/o randomization (of sample assignment)
- field
- combined
- lab
- (True) Experiment w randomization (of sample assignment)
- field
- combined
- lab
tradeoff of field VS lab experiments
realism –> generalizability = external validity
VS
control & ease of implementation –> internal validity
randomization:
object
goal n how
of assignment to one or the other condition;
the goal is to achieve internal validity because probabilistic assignment tends to produce similar populations
8=1+3+4 Threats to internal validity from extraneous variables
+ 1 important countermeasures
- 1 before exp:*
- selection bias = non-random assignments of treatments, eg doctors giving them to the neediest
- 3 experiment-related:*
-
socially desirable behavior (= wanting to look good) and/or demand effects (= giving researchers what they want)
>>> need to include measures of social desirability, coz this is usually the biggest problem in social R! - instrumentation = changes in instruments, observers (eg changing confederates in the exp.) or scores themselves
- testing effects = behavior changes due to test
4 time-related:
- history = specific events that happen at the same time
- maturation = changes influencing test units w time;
- similar to history, but more vague*
- mortality = loss of test units durign experiment
- regression to the mean = probabilistic issue of test units w extreme scores moving closer to avg during
>>> randomization helps against these effects
4 ways to control for extraneous variables
- randomization <<< best way, but not always possible
- matching = knowing the domain >>> find comparable pairs among test units >>> reduces sample size
- statistical control = measure & analyse confounding factor
- design control = add another experimental condition to manipulate
2 truly experimental designs
>>> possible problems
- pretest-posttest control group >>> testing effects cannot be excluded
- posttest-only control group >>> different populations at outset cannot be excluded
4 not truly experimental designs (in a table):
w differences wrt true exps
+ usually…
without control group:
- One-shot case study >>> no Control Group, no randomization
- One-group pretest-posttest design >>> no Control Group, no randomization
with control group:
- Static group design >>> no randomization
- Nonequivalent-groups (pretest-posttest) design >>> maybe multiple diffs bw experimental & control group
+ usually these are natural settings
Statistical experimental designs:
typical property, =
factorial design w several groups covering all the combinations of manipulated variables
