Threats to Internal Validity

Question 1

What are the three major threats to internal validity?

Answer 1

chance
bias
confounding

Question 2

What is chance?

Answer 2

random error
-inherent in all measurements
less random error=good
more random error=bad

Question 3

What is the statistical value we use to report chance?

Answer 3

p-value

Question 4

Describe the p-value and its relevance to chance.

Answer 4

probability representing the strength of evidence to support the null hypothesis
-p value > 0.05: supports null hypothesis
-p value < 0.05: reject the null hypothesis
probability the results are due to chance than a real treatment effect

Question 5

Differentiate between p-value of >0.05 and <0.05.

Answer 5

> 0.05: supports the null hypothesis
-no statistically significant difference between the groups results
<0.05: reject the null hypothesis
-there is statistically significant difference between the groups results

Question 6

How do we deal with chance?

Answer 6

increase the sample size
recognize the extent (through stats) and interpret the results accordingly

Question 7

True or false: statistical significance=clinical significance

Answer 7

false

Question 8

What is confounding?

Answer 8

when some factor(s) other than the intervention or exposure under study, influence the outcome

Question 9

True or false: we dont always measure or even recognize a confounder

Answer 9

true

Question 10

How do we deal with confounding?

Answer 10

randomization (gold standard)
-ensure groups are similar in all aspects (known and unknown factors)
stratification
matching (often done by sex and age)
statistical models

Question 11

Why is the table of baseline characteristics so important?

Answer 11

gives insight if the groups are similar and if the results will be applicable to other groups

Question 12

What is bias?

Answer 12

problems with the way a study was designed, conducted, or analyzed that leads to incorrect results or conclusions
-usually because the two groups were treated differently somehow

Question 13

Differentiate selection bias and information bias.

Answer 13

selection bias:
-systematic error (or differences in how the study subjects were selected or who participated
information bias:
-problems with measuring, collecting or classifying information

Question 14

What does selection bias primarily affect?

Answer 14

external validity

Question 15

What are some examples of selection bias?

Answer 15

self-selection/volunteer bias
-people who volunteer are different from those who dont
healthy worker (adherer) bias
-employed individuals are usually healthier
attrition bias (lost to follow-up)
-a concern if there are differences between those who are lost and those who werent

Question 16

Differentiate between outcome errors and exposure errors.

Answer 16

outcome errors:
-RCT and observational studies
-problems with measuring tools
-problems with actual measurements
exposure errors:
-observational studies
-problems with how subjects are categorized
-problems with measuring tools

Question 17

When is information bias a concern?

Answer 17

when the likelihood of being misclassified is unequal between groups

Question 18

What are some examples of information bias?

Answer 18

recall bias
-individuals remember things differently
-subjects with the outcome (esp if negative) are more likely to remember the intervention
interviewer bias
-interviewer asks about exposure/outcome differently
-leading, probing or influencing questions
-multiple interviewers
surveillance/detection bias
-one study group followed more closely than the other

Question 19

How do we minimize bias?

Answer 19

recognize and acknowledge it
-can never completely eliminate it, try minimize it
clear definition of study and sample population
treat all groups the same except for the intervention
standardized measurements
collect information from all subjects in the same way
BLINDING

Question 20

What is publication bias?

Answer 20

authors and journals tend to publish positive findings (esp with drug trials)
-may lead readers to believe there is association when its not the case