10.1: Internal & External Validity ✅ Flashcards
Interprofessional education quote
“2 or more professions learn about, from and with each other
-to enable effective collaboration and improve health outcomes”
Clinical and Public Health requires multi-disciplinary team
Internal Validity
- Are the results valid in terms of the truth in the source population?
- Exposure/outcome association or are the results explained by something else?
External Validity
- Can our results be generalized to the general population or to other similar populations?
Internal Validity
Our the results influenced by:
1) Chance
2) Bias
3) Confounding
Chance
Due to random nature of sampling, random error will be an important role in results
Any estimate is subject to chance
Any findings could potentially be a chance finding
This is determined by the p-value and 95% confidence interval
How do you determine if the validity of results affected by chance?
Check statistical significance
If statistically significant, in can be concluded with relative safety that the findings aren’t due to chance
How do you minimise the influence of chance
Take a large, representative sample to:
-reduce standard error and
-increase the power of the study
Selection bias
Errors in the process of sampling which result in selecting a non-representative sample
-and consequently any derived estimate will likely be biased (e.g., sampling bias, etc.)
Information bias
Errors in the process of data collection, which result in: -inaccurate assessment of the exposure and/or outcome variables
(e.g., recall bias, interviewer bias, etc.)
How do we determine if the results are affected by bias?
Not always easy to determine
Researcher needs to use their reflection and self-criticism
Samples using convenience sampling and measurements resulting from participant self-reports should always be expected to be suffering from bias
How to minimise the influence of bias?
Choose sample representative of the source (minimise selection bias)
Use tools that have high accuracy (minimise information bias)
Making a thorough investigation of the accuracy of the data collected (data cleaning)
What can Compounding greatly compromise?
The validity of a study
-by distorting the estimate of a potential association and even lead to masking a true association
-making a false association ‘appear’, when no true association exists.
How do we determine if results are affected by confounding?
Non-experimental conditions should expect to be affected
Consider confounding if an unexpected result or if there is something doesn’t make sense biologically
How to minimise the influence of confounding?
Make a list of potential confounders
Adjust for the potential confounders during data analysis
Be cautious of residual confounding
External validity
Interested in whether our findings can be generalized to:
1.The general population
2. Other similar populations
How to determine whether our results have low external validity?
Matter of correct judgment of the researcher
Whether sample is representative or not of the general population or similar populations in other countries
-going through the literature and investigating the evidence on the association of interest is always very helpful for evaluating the external validity of our findings
*we should never assume that our findings are externally valid without careful thought and investigation first
How to improve external validity?
Avoid highly selected samples if we want to generalize our results to the general population.
-choosing a random sample is the ideal approach If we have intentionally chosen a population with specific characteristics
Our sample accurately represents the specific population sub-group In such a case:
-generalizations to the general population should be avoided
-generalizations to other similar populations could be done but always under certain assumptions
Note on internal and external validity
-apart from one of the criteria of internal validity (chance)
Everything else related to judging the internal and external validity of a study is somewhat subjective in nature and requires:
-experience,
-critical thinking, and
-informed judgment from the researcher.
What must be done before deriving conclusion and generalizing research findings?
Thoroughly examine and confirm both internal and external validity of the relevant research studies
What must you do with evidence before you use it in clinical practice?
Examine the evidence critically
What happens when you work together with the multidisciplinary team?
Results in optimal patient and community care
