CAT: COHORT STUDIES CASP Flashcards
Clearly focussed question?
PICO
Was the cohort recruited in an acceptable way?
Was cohort representative of defined population?
Was there something special about cohort?
Was everybody included who should have been?
SELECTION BIAS
Examples from practice paper
- cohort should be representstaive of population
- selection of cohort will have particular important on external validity as observed associations may not be generalisable in different cohorts
Was the exposure accurately measured to minimise bias?
Subjective or objective measurements?
Do measurements truly reflect what you want them to (validated)?
We’re all subjects classified into exposure groups using the same procedure?
CLASSIFICATION BIAS
MEASUREMENT BIAS
Examples from practice paper on why inaccuracy in exposure measurement can lead to biased results:
- non-differential error - random misclassification - both groups equally affected and outcomes may be bias towards null and hide true relationships - type 2 error
- differential error - when 1 group has a greater proportion of errors - could bias in any direction creating spurious associations or obscuring true ones - type 1 error
Was the outcome accurately measured to minimise bias?
Subjective or objective measurements?
Do measurements truly reflect what you want them to (validated)?
Has a reliable system been established for detecting all cases?
We’re measurement methods similar in different groups?
We’re subjects/outcome assessor blinded to exposure (does this matter)?
CLASSIFICATION BIAS
MEASUREMENT BIAS
Examples from practice paper on why inaccuracy in outcome measurement could lead to biased results:
- if the outcome measurement reporting is uniformly poor and disease outcomes are missed then we see a tend towards null - type 2 error
- if the outcome measurements are more accurate in 1 exposure group we may get a spurious positive relationship (type 1 error)
Example from practice paper on features of study method that can enhance the accuracy of outcome measurement:
- standard application of outcome measurement
- multiple and independant asessors
- validation of algorithms used
- comprehensive population coverage
- outcome established blind to exposure category
Have the authors identified all important confounding factors?
Have they taken account of the confounding factors in the designs and/or analysis?
List any you think are important and ones they missed
Look for restriction in design
Look for techniques e.g. modelling, stratified, regression, sensitivity analysis
Example from practice paper on the consequence if all important confounders are not accounted for:
- association between exposure and outcome could be biased
Was the follow up of the subjects complete enough?
Was the follow up of the subjects long enough?
Was it long enough for good and bad effects to reveal themselves
Persons lost to follow up may have different outcomes than those available for assessment
Was thee anything special about the outcome of the people leaving or the exposure of the people entering?
Example from practice paper on why the completemnes and length of follow-up may influence validity:
- incomplete follow up allows for selection bias
- incomplete follow up may reduce the power of the study
- differential follow up will lead to bias
- insufficient length may mean relevant outcomes are not observed and associations are not detected
What are the results of the study?
Bottoms line results
Rate or proportion between exposed/unexposed, the ratio/rate difference
Relative risk - how strong is the association between exposure and outcome
What is ARR
How precise are the results?
Look for the range of CI
Does CI exclude the null relationship?
Example from practice paper on how you can increase precision:
- increased number of events through longer follow-up or larger number of participants
Do you believe the results?
Big effect is hard to ignore
Can it be due to bias, chance or confounding?
Are the design and methods of the study sufficiently flawed to make results unreliable?
Bradford hills criteria
Can the results be applied to the local population?
Was a cohort study appropriate?
Were subjects sufficiently different from your pop to cause concern?
Is your local setting likely to differ much from that of te study?
Benefits and harms
Example from practice paper:
- look for description of study cohort to allow comparison with local population
- consider whether exposure pattern was likely to be similar to local population
- information to allow assessment of internal validity as if not valid then not possible to apply externally
What are the implications of this study for practice?
1 observational study rarely provides sufficiently robust evidence to recommend changes to clinical practice
Recommendations are stronger when supported by other evidence
What are examples of selection bias?
Sampling bias - some members of the intended population are less likely to be included than others (non-random sampling)
Attrition bias - participants who drop out of a study semantically differ from the ones who remain
Volunteer bias - people with specific characteristics are more likely to participate than others
Non-response bias - people who refuse to participate systemically differ from those who took part
How does selection bias affect validity?
External - results from biased sample may not be generalisable to population as sample isnt representative to target pppylation
Internal - inaccurate estimate of relationships between variables
