Understanding Research Evidence Flashcards
What type of evidence do we look for?
Peer-reviewed publications and/or systematic reviews.
Study design (PICO) must be relevant for what is being looked for. Authors sometimes incorrectly define their designs
Consider possible study limitations and bias
Statistical power
What is a randomised controlled trial?
An experimental study (not observational) where peoeple are randomly allocated.
What are the types of observational studies?
If there is no comparison group it is a descriptive study.
If there is a comparison group:
If exposure -> outcome it is a cohort study (eg what has eating apples done to people)
If outcome ->exposure it is a case control study (what is causing these strokes?)
If exposure and outcome are at the same time it is a cross-sectional study
What are the benefits of conducting a randomised controlled trial?
The gold standard in clinical research
Designed specifically to minimise bias
Is the equivalent of a controlled experiment in basic science
If properly designed it can be free of bias and thus statistically robust
What is a randomised controlled trial?
Participants are randomized into 2 or more groups and each group receives a different intervention.
Assignment of treatment occurs “by chance”
Participants and staff both don’t know what they are giving the patient, the treatment or the placebo.
What research questions can be addressed by a randomised controlled trial?
Interventional questions; is the treatment effective? Is treatment A more or less effective than treatment B?
What are the disadvantages of conducting a randomised controlled trial?
While it has internal validity, having exclusion and inclusion criteria may mean it doesn’t work for people outside this cohort.
Sometimes, can’t be done due to ethical concerns
Can be expensive
How is bias eliminated in an RCT?
Blinding of participants and staff
Placebo controlled
What is the difference between passive and active placebo?
Passive placebo - mimics administration only
Active placebo - mimics side effects of intervention
What is a cohort study?
An observational study where participants are followed over time (Eg effect of birthweight on future development of heart disease)
Participants with specific characteristics are a “cohort”
Differences between them are measured and they are followed over time.
What are cohort studies used for?
Determining risk factors/predictors of risk
Aetiology (What causes the outcomes?)
Prognosis (What happens in this disease over time?)
Diagnosis (If test x is positive, what happens to the patient?)
What are the pros and cons to cohort studies?
Pros: Best way to identify incidence and relative risk of a disease
Usually a clear view of the exposure-outcome timeline
Can investigate multiple outcomes
Useful to study rare exposures
Cons: Assessing rare events or events that take a long time to develop may be too slow to yield results
Selection bias - hard to match exposure and non-exposure groups exactly
Not a great design for rare diseases
Loss to follow-up can lead to bias (hard to follow everyone in the cohort due to random reasons)
Exposure status might change over time
What is a case-control study?
Observational study comparing 2 matched groups of participants; 1 has the outcome, the other doesn’t. The study analyses the differences and identifies risk factors
What questions can case-control studies answer?
Risks
Diagnosis
Prognosis
What are the pros and cons of case-control studies?
Pros: Most-widely used study design
Simplest to execute
Quick to perform
Cheap
Cons:
More susceptible to bias
Easy to do but can be done incorrectly
How is the control group for a case-control study selected?
Control group should be free of the outcome
Represent individuals selected as cases
Selection of the controls MUST be independent of the exposure
What are cross-sectional studies?
Measure a population at a particular point in time in order to detect a particular outcome
What questions can cross-sectional studies answer?
Frequency - how common is the outcome?
Aetiology - What risk factors are associated with the outcome
Diagnosis - Does the new tst perform as well as the current standard test?
What information can’t cross-sectional studies provide us with?
Can’t determine relationship between outcome and exposure
All that can be said is that at a particular point in time obese people had a higher prevalence of arthritis but we can’t make a definitive relationship between arthritis and obesity.
What are qualitative studies?
Name given to group of designs which focus on patient oriented outcomes
Qualitiative studies are very systematic in their approach but they also ask questions like:
Why do people….
What are the reasons for…
How do people feel about….
They use interviews, focus groups, and participant observation.
What type of strategies are used for qualitative studies?
Involve explicit sampling strategies and systematic data analysis
Portray the voice of the participant, usually absent from qualitative studies and therefore provides complementary information
What is the value of qualitative studies if they are so subjective in nature?
There is value in understanding personal experience and how it impacts the patient and practitioner experience
These methods can bridge the gap between scientific evidence and clinical practive and provide additional improvements to patient care.
What kinds of problems can qualitative studies help solve?
Patient compliance/adherence with medication
Disparities between how doctors and patients view medical conditions and treatments
What is a case study or case-series study?
A study describing a patient or small group of patients affected by a specific disease or exposed to a factor
What are case-studies used for?
To create hyptheses and lead to larger incidences of research.
What information does good descriptive research provide us with?
Who is affected
Outcome
Why did this outcome occur?
When did the outcome arise?
Where did this happen?
Who, what, when, where, why, how?
What are the pros and cons of case studies?
Pros:
One case to initiate a signal
Provide stronger evidence with multiple cases
Observational
Educational
Easy to do
Identify rare manifestations of a disease or drug
Cons:
No control
Difficult to compare different cases
Cases may not be generalizable
Selection bias
Unknown future outcome/follow-up
How are case studies used?
They do not rank highly in evidence hierarchy
Least publishable due to small number of patients
but the information can sometimes be important to provide a basis for hypothesis to be built
How are study designs chosen?
Based on the type of question being answered:
Example:
(Prevalence is answered with a cross sectional study
Incidence, cause and prognosis are answered with cohort studies.
Treatment effect is tested with controlled trial)
What is the hierarchy of evidence for the questions?
Intervention: RCT > cohort > case control > Case series
Diagnosis: Blind comparison to gold standard
Aetiology: RCT > Cohort > Case control > Case series
Prognosis: Cohort > Case control > case series
Prevention: RCT > Cohort study > Case control > Case series
How is the hierarchy of evidence ranked?
Using absence of control or comparison groups
What is the strongest form of evidence for making clinical decisions?
Clinical practice guidelines which are made using systematic reviews and meta analyses
What evidence must we look for to use in clinical decisions?
The best evidence to use in decisions is the evidence
highest in the hierarchy. Evidence from a lower level
should be used only if there is no good randomised
controlled trial to answer a particular clinical question
What is the hierarchy of evidence based on?
Risk of bias
What is the NMHRC hierarchy of evidence levels?
I = evidence from systematic review of relevant RCT
II = Evidence from at least one RCT
III-1 = Evidence from well-designed pseudo-randomised controlled trials
III-2 = Evidence from comparative studies non-RCTs, cohort studies, case-control studies
III-3 = Evidence from comparative studies (no controls)
IV = Evidence from case series, case reports
What are some important challenges to the hierarchy of evidence?
Study design as a definition isn’t always a reliable indicator of the level of risk of bias
There should be flexibility to upgrade or down-grade evidence levels based on quality of evidence
Instead evaluating credibility of systematic reviews and meta-analyses is a better way to use them
How should RCTs be considered?
Although it is important to understand the strengths
and limitations of both RCTs (efficacy studies) and
observational studies (effectiveness studies), none of the
study designs should be considered in isolation since all
types of evidence rely primarily on the rigour with which
individual studies were conducted (regardless of the
methodological approach) and the care with which they
are interpreted
