WEEK 3- 5.2 threats to validity

Question 1

internal validity

Answer 1

ability to draw causal inferences
confidence in observed causal relationship
ability to rule out alternative explanations

Question 1

internal validity

Answer 1

ability to draw causal inferences
confidence in observed causal relationship
ability to rule out alternative explanations

Question 2

external validity

Answer 2

generalizability of research findings
closely linked to case selection and sampling

Question 3

reliability

Answer 3

consistency and replicability of findings
will you get the same outcomes again?

Question 4

common threats to (mostly) internal validity

Answer 4

1. history (context effect):we want to see the effects of economic stimulus package, but covid-19 led to a shutdown (external events having a powerful impact)
2. maturation (subject effect): biological and psychological changes. long term effects in longtidunial studies
3. testing/ performance (sensitization): if the same person is interviewed in two or more occasions participants learn or questions provide cues for the next one, or participants getting tired
   => artificial effects, introduced by the design or method possible bias
   4.** instabilty/ instrumentation** (effectiveness, bias) problem that the method of measurement might lead to different responses over time. if a survey question changes, or the response scales/options change. is the outcome due to opinion change, or phrasing?

Question 5

what’s history (context) as a threat?

Answer 5

threat to internal validity
:we want to see the effects of economic stimulus package, but covid-19 led to a shutdown (external events having a powerful impact)

Question 6

what’s maturation (subject effect) as a threat?

Answer 6

threat to internal validity
biological and psychological changes. long term effects in longtidunial studies

Question 7

what’s testing/ performance (sensitization) as a threat?

Answer 7

threat to internal validity
if the same person is interviewed in two or more occasions participants learn or questions provide cues for the next one, or participants getting tired

Question 8

what’s instability/ instrumentation (effectiveness, bias) as a threat?

Answer 8

threat to internal validity
problem that the method of measurement might lead to different responses over time. if a survey question changes, or the response scales/options change. is the outcome due to opinion change, or phrasing?

Question 9

what are threats to (mostly) external validity?

Answer 9

1. selection bias/ generalizability: case selection, randomization ,self selection for sampling, and random asignment or proffesional participants
2. statistical regression- regression toward the mean. for example, you want to research people who are very politically knowledgeable. in the first test, some people will by accident score very high, and some will accidentally score low => pre selection based on performance may be biased.
3. statistical regression- ceiling and floor effects. if the outcome measure is already very high or very floor, it cannot further increase or decrease anymore. for example, if we study agenda setting for climate change, and if its already at the top in the agenda it willnot increase anymore
4. mortality (attrition): participants in longtidunial panel mostly drop out or die. original number shrinks over time, and if the one’s who drop are all politically not interested, the sample becomes less representative, and more biased.**

Question 10

what’s statistical regression- regression towards the mean as a threat?

Answer 10

threat to external validity
for example, you want to research people who are very politically knowledgeable. in the first test, some people will by accident score very high, and some will accidentally score low => pre selection based on performance lead to bias

Question 11

what’s statistical regression- ceiling and floor effects as a threat?

Answer 11

threat to external validity
if the outcome measure is already very high or very floor, it cannot further increase or decrease anymore. for example, if we study agenda setting for climate change, and if its already at the top in the agenda it will not increase anymore

Question 12

what’s mortality (attrition) as a threat?

Answer 12

threat to external validity
participants in longtidunial panel mostly drop out or die. original number shrinks over time, and if the one’s who drop are all politically not interested, the sample becomes less representative, and more biased.