Research Designs Flashcards
true or false: the only method of control over pre-existing differences is through dispersion.
True
What are major REASONS for choosing a within-subjects design?
- Dispersion; variables are spread equally because they are compared to themselves
- greater statistical power to find small effects
- Convenience (less people needed)
Explain the statistical reason for why you would opt to do a W-S experiment?
Lower Type 2 error (i.e. more likely to reveal a significant difference if a difference does in fact exist)
Maximizes primary variance associated w/ IV & reduces unwanted error variance (NOISE)
When would you choose to not do an W-S experiment and go for between-subjects?
- If conditions cause lasting change; can’t give other levels of the IV because permanent change has happened
- Convenience reasons (lengthier time in session or multiple times)
What are the major threats to the internal validity of a W-S?
All are related to possible effects of repeated testing
Order Effects: response changes in a systematic fashion
Carry Over Effect: when a response to one task/condition varies as a function of whether another task/condition that precedes (or follows) it
A child has to do both a lacing task and puzzle task. By the time the child gets to the puzzle, the performance level starts decreasing. What time of order effect is this? (A THREAT TO INTERNAL VAL.)
Fatigue effect
A child has to do both a clean up task and a reading task. The child did not perform well on the clean up task, but as they got more familiar, their performance increased. What type of order effect is this? (A THREAT TO INTERNAL VALIDITY)
Warm-up effect
A child is presented with a cognitive test that has them do both an inhibitory control test and an attentional span test. Although the child believed that the strategy to the 1st task would be applicable to the 2nd, it was not. What type of threat to internal validity is this?
Carry-Over effect
True or False: There is no general improvement or decline in carry-over effects
True
When are the internal validity threats most likely in W-S experiments?
When researcher adopts a constant order of presentation of conditions
What are the alternatives to a consistent order in administration of conditions?
Randomize (good for when # of tasks/conditions is large)
Counterbalancing (distribute task/condition equiv. across all possible positions)
Which is better counter-balancing or randomizing?
Counterbalancing bc it ensures no confounding of task & order
Permits check of order effects (if ea order is represented enough, we can compare the diff orders of pres)
When would you HAVE TO CHOOSE a between-subjects design?
If expect conditions to cause lasting change.= (e.g. Therapy)
What is the MAJOR THREAT to internal validity in a B-S design?
When the two IV groups have systematically different kinds of participants in them = NON EQUIVALENT GROUPS
What is the selection effects/bias?
When there are already differences found between condition groups; thus not a true effect of experimental manipulations; NOT RANDOMLY DISPERSED
To rule out pre-existing differences between participants in different conditions/groups, the
experimental method of control is through
equal dispersion = meaningful human differences are spread equally across the different conditions/groups.
How do we rule out the non-equivalent groups threat in B-S design?
Random assignment
matching on important attributes
True or False: It’s common for random assignment to be done in combination with a matching constraint (e.g., sex of infant); researchers seldom use totally random assignment.
True
True or False:Most B-S studies include a degree of matching for variables like age and sex; this is a wider attempt to match on all variables of potential importance.
True
What is the logic behind RA?
If ea. person has an equal chance of being selected, then the characteristics associated w/ each person have an equal chance of falling in each group
How does matching on important attributes work (aka Matched Groups)
Identifies dimensions on which to make groups equal (e.g., IQ, SES, health status - the list will vary across studies) and then assigns participants based on these dimensions.
True or false: Matching is always necessarily “partial matching”
true; we can never identify all potentially important variables
What remains more important even in a matched-group design?
The random assignment
What is the defining characteristic of a correlation design?
no manipulation of causal variable (i.e., IV/predictor); just measured as it naturally occurs
What does a correlational statistic measure?
degree of relation between two variables (i.e., the relation between “predictor” & “criterion”); how much the variables “move” up and down together; THE NUMBER (-1 to +1)
Direction of relation; whether variables “move” in the same direction (positive +) or in opposite directions (i.e., when one goes up, the other goes down) (negative -)
True or False:A correlational design and a correlational statistic are separable.
True
True or False: Statistics other than correlation statistic/coefficient can be used to examine results of correlational design
True
True or False: In some cases, a certain type of corr. coefficent can be calculated for experimental designs
True
Correlational design = ?
“Nonexperimental” design (better phrasing)
group/condition comparisons based on correlational data
Why would you choose to do a non-experimental design?
- wider range of variation/levels of the IV/predictor variable
- manipulation of IV/predictor is impossible (e.g. parent-child relationship quality)
- manipulation is unethical (e.g. spanking)
- manipulation is difficult (e.g. experiment w/in natural setting or too aritficial)
Why can’t we establish cause-and-effect in a correlational/experimental design?
Lacks all of the forms of control we talked about earlier; control over the iv, setting, and pre-existing differences
Lack of control of IV/predictor means lack of?
Internal validity
What is the reverse causality problem?
Unsure of the direction of the effect ; X-> Y or Y->X
How can we rule out reverse causation?
Nature of the variables; body size may affect aggression, but aggression may not cause your body size
When does the reverse causality problem happen?
Sometimes
What is the “third variable” problem?
Can’t be sure of equivalent comparison groups (i.e., can’t rule out a systematic difference on an extraneous variable; a preexisting difference or during study
When is the third variable problem a problem?
Always
How can we strengthen causal inferences for reverse causality?
Measure variables at more than one time, then calculate cross-lag correlations
For reverse causality; if X causes Y; what should correlate and what shouldn’t?
X1 & Y2 should sig correlate
Y1 & X2 should NOT correlate
true of false: the correlation test for reverse causation proves causal role of X
False; it does not bc the study does not have exp. control, but does heighten probability of causal inference
True or False: We ALWAYS need to address the confound variable in correlational studies
Yes; major threat of C
How do we address the confound variables?
1) measure potential C
2) plan to statistically test for whether C varies with your IV and most importantly, your DV
3. if PV is cont & has no groups (also assum. C is cont) do a regression analysis/ if P has group/cat (C should be cont); then do t-test or ANOVA to see if means/avgs differ
4. since most DVs (Y) are cont, you test whether c systematically varies with your DV w/ a corr coefficent
How do we address the confound variables?
1) measure potential C
2) plan to statistically test for whether C varies with your IV and most importantly, your DV
- if PV is cont & has no groups (also assum. C is cont) do a regression analysis/ if P has group/cat (C should be cont); then do t-test or ANOVA to see if means/avgs differ
- since most DVs (Y) are cont, you test whether c systematically varies with your DV w/ a corr coefficent
- Check results; C is only a confound if it varies with BOTH PV/IV and DV
C is only a confound variable when what?
Varies w both DV and IV/PV
How do you address “third variable problem”
By stat controlling/removing confound factors
What do you do when there is a confound involving CONTINUOUS variables?
include covariates in multiple regression; whether X1 or X2 predicted any variation in the DV above and beyond explained by the confound C `
What do you do when there is a confound involving groups?
include C as a covariate when conducting a one-way or two-way ANOVA (i.e., ANCOVA), then see if the means of your groups significantly different
If you study a SINGLE predictor (X1) that is continuous there’s another option…
compute partial correlation for y-X1
What does partialing out do?
removes the contribution of a third variable C from a correlation between two other variables (Y, X1).
Partial- correlation holds the C ____ while examining the relation between the two variables
constant
What is the main limitation of statistical control in non-experimental designs?
No way to identify/measure, and statistically remove all possible confounding C factors.
