Threats to Internal Validity

Question 1

What are the three main threats to internal validity?

Answer 1

1. Bias
2. Chance
3. Confounding

Question 2

What is chance?

Answer 2

It is the random error that is inherent in all measurement

Less random error = good

Chance can be reduced by broadening sample size

Can be estimated with statistics (p-value)

Question 3

What is p-value?

Answer 3

It is the probability representing the strength of evidence to support the null hypothesis (no statistical difference between groups)

Question 4

What does a large p-value indicate in relation to statistical significance?

Answer 4

It supports the null hypothesis (no statistical difference between groups)

Chance is higher

Question 5

What does a small p-value indicate in relation to statistical significance?

Answer 5

It rejects the null hypothesis (there is a statistically significant difference between groups)

Chance is lower

Question 6

How can researchers manage the effects of chance on their studies?

Answer 6

1. Increase sample size
2. Recognize extent of chance via statistics and interpret the results accordingly

Question 7

Are all statistically significant findings, clinically significant?

Answer 7

No, some statistical significant differences are so small, that it is often not worth making those changes clinically

ex. a treatment shows to extend patient survival by 10 days, but it is likely not worth the increased side effects, suffering, and drug costs

Question 8

What is confounding?

Answer 8

When some factor(s), other than the intervention or exposure under study, influence the outcome

ex. coffee drinkers have heart attacks more, but if you look at more data, it seems like coffee drinkers are more likely to smoke vs non-coffee drinkers. The confounding variable is smoking status

Question 9

What are some potential confounders?

Answer 9

The following is not an exhaustive list:
Age
Level of exercise
Diet
BP
Sex
medication history

Question 10

How can confounding variables be managed?

Answer 10

1. Randomization (ensures groups are similar in all aspects)
2. Stratification (minimize differences in specific demographics)
3. Matching (similar types of subjects in both groups)
4. Statistical models (try to account for the effects of known confounding variables, but do not adjust for unknown variables)

Question 11

What is bias in a research study?

Answer 11

These are problems with the way a study was designed, conducted, or analyzed that lead to incorrect results or conclusions (usually due to differential treatment between groups)

Question 12

What are the two main types of bias in research?

Answer 12

1. Selection bias
2. Information Bias

Question 13

What is selection bias?

Answer 13

There are problems with how the study subjects were selected (lacking standardization)

Question 14

What are some types of selection biases seen in research?

Answer 14

1. Self-selection bias (people who participate in research study are different compared to people who do not participate)
2. Healthy worker (adherer) bias (Participants with higher socio-economic situations are usually healthier than the general population)
3. Attrition bias (lost to follow-up): (participants leaving the study can create differences between groups and increase chance)

Question 15

What is information bias?

Answer 15

Systematic errors in the way subjects were measured or classified

Question 16

What are the two types of information biases?

Answer 16

1. Outcome errors (RCT and observational studies)
2. Exposure errors (more with observational studies)

Question 17

What are outcome errors?

Answer 17

Problems with actual measurements:

Problems with measuring tools:

ex. one group measured with BP machine and one group had BP measured manually

Question 18

What are exposure errors?

Answer 18

Problems with how subjects are catagorized

Problems with measuring tools

ex. patient in no Vitamin D group takes a multivitamin with Vitamin D included

Question 19

What is recall bias?

Answer 19

Individuals remember negative experiences more vividly and are able to better recall events in the past

Question 20

What is interviewer bias?

Answer 20

Interviewer asks about exposure/outcome differently

Leading, probing, or influencing questions

Multiple interviewers may have a slightly different approach

Question 21

What is surveillance bias?

Answer 21

One study group is followed more closely than the other

More outcomes are recorded because of more follow-up, but not necessarily the actual incidence rate was higher

Question 22

How can bias in research studies be minimized?

Answer 22

1. Clear definition of study and sample population
2. Ensure all groups are treated the same except for the intervention
3. Standardized measurement (same questionnaire, interviewers, devices, and labs)
4. Blinding (single vs. double-blind studies)