Week 10 - Internal and External Study Validity

Question 1

What does The World Health Organisation say about Interprofessional Education (IPE)?

Answer 1

Interprofessional education = occurs when two or more professions learn about, from and with each other to enable effective collaboration and improve health outcomes

Question 2

What is internal validity?

Answer 2

Internal validity means there is an exposure-outcome association in the source population.
The results cannot be explained by anything else.

Question 3

What is external validity?

Answer 3

External validity is when the results of the sample can be generalised to the general population or other similar populations.

Question 4

What 3 factors can influence internal validity?

Answer 4

1. Chance (random/ sampling error)
2. Bias (systematic error)
3. Cofounding

Question 5

How is Chance an influence on internal validity?

Answer 5

• Random nature of sampling thus chance will always play an important role in our results, because we use sample to derive conclusions about populations
• Any estimate from any study is a subject to chance (ANY FINDING CAN BE A CHANCE FINDING)
• The role of chance is determined by the p-value and 95% Confidence Interval

Question 6

How do we determine whether validity was affected by chance?

Answer 6

1. Check findings for statistical significance (p-value/ 95% Confidence Interval)
2. If the exposure-outcome association is statistically significant. THUS NOT DUE TO CHANCE

Question 7

How can we minimise the influence of chance in our findings?

Answer 7

1. Large, representative sample must be used
2. Thus reduces standard error and increases study power THUS MORE PRECISE ESTIMATES

Question 8

What are the two types of bias?

Answer 8

1. Selection Bias
2. Information bias

Question 9

What is selection bias?

Answer 9

-Errors in sampling
-Creates a non-representative sample
-Any derived estimate is likely biased

Question 10

What is information bias?

Answer 10

-Errors in process of data collection
-Inaccurate assessment of exposure/ outcome variables (recall bias, interviewer bias etc.)

Question 11

How do determine if the validity is affected by bias?

Answer 11

-Difficult to determine
- Researchers need to make his/her reflection and self-criticism regarding appropriateness of sample chosen and accuracy of measurements
- SAMPLES FROM CONVENIENCE SAMPLING AND SELF-REPORTS ALWAYS ARE EXPECTED TO CONTAIN BIAS

Question 12

How can we minimise the influence of bias on our findings?

Answer 12

1. Choose representative sample (lowers selection bias)
2. Choose assessment tools having high accuracy (minimises information bias)
3. Make thorough investigation of accuracy of data collected (must be done during data collection AND data analysis (data cleaning))

Question 13

What is confounding in Internal Validity terms?

Answer 13

• Confounding = a factor explains all or part of association between exposure and outcome
  –> Compromises validity ( distorts the estimate of possible association
  –> Can mask true association
  –> OR makes false association appear

Question 14

How do we determine if validity is affected by confounding?

Answer 14

• Under non-experimental conditions = expect that confounding is affecting results to some degree
• Confounding effects includes:
  –> unexpected result is obtained
  –> OR something that doesn’t make sense biologically

Question 15

How can we minimise the influence of confounding in our findings?

Answer 15

1. Make a list of potential confounders for given exposure-outcome association
2. Adjust for potential confounders during data analysis (cofounder-adjusted estimates which are free of confounding effects)
3. ALWAYS BE CAUTIOUS OF RESIDUAL CONFOUNDING

Question 16

In external validity where do we wanna generalise our findings to?

Answer 16

1. General population
2. Other similar populations

Question 17

How do we determine whether results have low external validity?

Answer 17

• Depends on correct judgement of researcher
• Based on whether study sample is representative of general populations or similar populations in other countries
• Going through the literature and investigating evidence on association helps determine external validity
• NEVER ASSUME EXTERNAL VALIDITY WITHOUT INVESTIGATION FIRST

Question 18

How do we improve the external validity of our results?

Answer 18

1. We should AVOID highly selected samples if we want to generalise sample (choosing random sample is ideal)
2. If we intentionally choose a population with specific characteristics then we must ensure that our sample accurately represents specific population sub-group
   –> Avoid generalisation
   –> Generalisation to other similar populations can be done but ALWAYS UNDER CERTAIN ASSUMPTIONS

Question 19

What is the only criteria of internal validity that doesn’t depend on judging (subjective)?

Answer 19

CHANCE