Science of Evidence Based Medicine

Question 1

Identify the main study types. What type of research is this ?

Answer 1

1) Observational Studies
- Case control
- Cohort
- Cross-sectional

2) Experimental Studies
- Randomised and non-randomised trials

This is all primary research.

Question 2

Identify the main review types. What type of research is this ?

Answer 2

Expert Review
Systematic Review
Meta-analysis

This is all secondary research.

Question 3

Describe the components of the evidence pyramid.

Answer 3

From bottom to top:

• Personal Communication
• Case series and case studies
• Cross-sectional studies
• Case-control studies
• Cohort studies
• Non-randomised experimental designs
• RCTs
• Meta-analyses and systematic reviews

Question 4

How can we quantify levels of evidence in the evidence pyramid ?

Answer 4

Through SIGN levels of evidence.

Question 5

Describe the main features of case series, case studies and case reports.

Answer 5

In all three, looking at patient notes from a or a group of patients and writing paper about it.

Case study is about just one patient (e.g. rare disorder so rare amount of patients to study, or from remarkable cases which stand out)

Case series is about more than one patient.

Question 6

What are the main pros and cons of case series/studies/reports ?

Answer 6

PROS

• Quick and cheap to conduct
• Can be published quickly
• Early indicators of problems
• Can help detect new drug side effects and potential uses (adverse or beneficial)

CONS

• Very small number of patients
• No control group
• Statistically weak
• Cases may not be generalisable to the general population

Question 7

Describe the main features of cross-sectional studies/surveys.

Answer 7

Measures the outcome and the exposures in the study participants at the same time. (asking whether association).
E.g. target population diabetic, wanting to see if relationship between diabetes and having ischemic heart disease.

Question 8

What are the main pros and cons of cross-sectional studies/surveys ?

Answer 8

PROS:

• Ethically safe
• Cheap and simple
• Useful for planning purposes

CONS:

• Correlation does not imply causation (possible confounding variables)
• Volunteer bias (only those who fit with target group will be included)
• Unequal distribution of confounders

Question 9

Define confounders.

Answer 9

They are uncontrolled extraneous variables, i.e. not studied as part of the study
e.g. Smokers tend to have smaller babies than non-smokers. Confounding is alcohol because smokers tend to also drink more which may effect size of baby.

Question 10

Define spurious association.

Answer 10

“Two or more events or variables are not causally related to each other, yet it may be wrongly inferred that they are, due to either coincidence or the presence of a confounding variable”

e.g. Initial studies showed that babies exposed to ultrasound had lower birth weights but later observations failed to confirm this. Probably because US was used for babies with problems rather than for the whole population.

Question 11

Describe the main features of case control studies.

Answer 11

“Compares patients who have a disease or outcome of interest (cases, e.g. death, disease) with patients who do not have the disease or outcome (controls), and looks back retrospectively to compare how frequently the exposure to a risk factor is present in each group to determine the relationship between the risk factor and the disease.” Data on exposure is collected from medical records or the administration of questionnaires.

Question 12

What are the main pros and cons of case control studies ?

Answer 12

PROS:

• Looks at numerous risk factors simultaneously
• Can give early indication of association between disease and risk factor
• Does not require long follow up so unlikely to lose patients
• Good for studying rare diseases/conditions

CONS:

• Correlation does not imply causation (i.e. confoundings)
• Retrospective study so data collection on exposure requires patient recall (recall bias) or access to medical records (may not be able to access)
• Selection of adequate control can be difficult (same age, sex distribution etc.)
• Unlikely to detect rare causes of disease

Question 13

Describe the main features of a cohort study.

Answer 13

Prospective study in which outcome or disease-free study population is first identified by the exposure or event of interest (e.g. smoking) and followed in time until the disease or outcome of interest occurs. Then, determine which initial participants exposure characteristics (risk factors) are associated with outcome of interest.

Question 14

What are the main and cons of cohort studies ?

Answer 14

PROS

• Prospective so can show that cause precedes effect
• Ethically safe (but if outcome arises as a result of risk factor during study, need to give advice to patients)
• Cheaper and easier than RCT
• Subjects can be matched (age, sex…)

CONS

• Exposure may be linked to hidden confounding
• Drop out rate relatively high (follow up must be as high as possible)
• Blinding subjects is difficult
• Outcome of interest may take a long time to occur

Question 15

Describe the main features of a RCT.

Answer 15

“Randomly assigns participants into an experimental group or a control group. As the study is conducted, the only expected difference between the control and experimental groups in a randomized controlled trial (RCT) is the outcome variable being studied.”

Question 16

Explain what is meant by double blinding. What are the risks if an RCT is conducted without double blinding ?

Answer 16

Experimental procedure in which neither the subjects nor the experimenters know which subjects are in the test and control groups during the actual course of the experiments
Only person who knows is study manager.

Otherwise patients may have reporting bias (want drug to work) and doctors may treat subjects in control and treatment groups differently resulting in more bias.

Question 17

Identify and define the main design modifiers.

Answer 17

• Single blind- Patients did not know which treatment they were receiving
• Double blind- Neither patient not doctor know which treatment patients are receiving
• Crossover- Patients receive both the intervention and control treatment randomly, often separated by washout period. Gives internal control.
• Placebo controlled- Control patient receives placebo (i.e. inactive pill)

Question 18

If a placebo is not given, what else may be given to the control group ?

Answer 18

Control group given older version of medication, X while treatment group given new, experimental version of drug, Y

Question 19

What are the pros and cons of RCTs ?

Answer 19

PROS

• Unbiased distribution of confounders (same in both treatment and control groups)
• Randomisation helps systematic analysis
• Patients more likely to be blinded (so less likely for reporting bias + bias due to way doctor treats patient)
• Clearly identified populations

CONS

• Ethical issues if treatment groups seen to respond better or worse than expected
• Expensive (time and money)
• Volunteer bias (population may not be representative)

Question 20

Identify and define some design features of clinical research, and explain how analysis occurs for each.

Answer 20

1) Parallel group comparisons: Each group given a different treatment simultaneously. Analysis is by comparing groups.
2) Paired (matched) comparisons: Subjects matched to balance confounders (age, sex). Analysis is by looking for differences by subject pairs.
3) Within subject comparison: Patient assessed before and after given intervention. Analysing within subject changes.

Question 21

Identify possible problems with RCTs.

Answer 21

♠ Resources - the costs of RCTs are substantial in money, time and energy
♠ Stopping trials early - interim analyses of trials are now commonly undertaken to assess whether the treatment is showing benefit and if the trial can be stopped early
♠ Unnecessary - when a treatment produces a ‘dramatic’ benefit - imatinib (Glivec) for chronic myeloid leukaemia
♠ Generalisability - RCTs are often carried out on specific types of patients for a relatively short period of time
♠ Impossible - with treatments for very rare diseases where the number of patients is too limited

Question 22

What are reasons for a correlation besides a causation (i.e. why might two factors be seemingly correlated without there being a causal relationship between the two) ?

Answer 22

1) The association is due to chance
2) The methodology is flawed (bias)
3) Confounding factor linked to both
the exposure and the outcome

Question 23

Define the Bradford Hill criterion.

Answer 23

“Criterion to provide evidence that a causal relationship exists between two factors”.
Initially meant for smoking and lung cancer.

Question 24

What are The Bradford Hill criterion ? Apply these to cigarette smoking and lung cancer.

Answer 24

Biological Plausibility- A causal link is more likely if a biologically plausible mechanism is likely or demonstrated (e.g. Cigarette smoke contains many compounds known, or since shown, to be carcinogenic)

Consistency- the association is observed in different studies and different sub-groups

Dose-response relationship- different levels of exposure to the putative factor lead to different risk of acquiring the outcome (e.g. more you smoke the more likely you are to develop lung cancer)

Reversibility- removal or prevention of the putative factor leads to a reduced or non-existent risk of acquiring the outcome (e.g. those who stop smoking reduce their risk over time)

Strength of association- causal link is more likely with strong associations (e.gvery strong relationship between smoking and the development of lung cancer)

Temporality- exposure to the putative cause has been shown to precede the outcome (e.g. Lung cancer may take 20 years to develop; increases in lung cancer start to increase 20 years after the prevalence of smoking increases)

Question 25

What are the main pros and cons of Expert Reviews ?

Answer 25

PROS:

• Comprehensive survey
• Answers a specific question

CONS:
-Expert bias (may have pre-formed opinion, selection of sources may not be systematic)

Question 26

Describe the main features of a systematic review.

Answer 26

Attempts to identify, appraise (notably through assessment of risk of bias) and synthesize all the empirical evidence that meets pre- specified eligibility criteria (inclusion and exclusion criteria) to answer a given research question.
Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform
decision making.

Question 27

Why are systematic reviews viewed as

the ’gold standard’?

Answer 27

Because they avoid or minimise bias.

Question 28

Identify and define examples of bias one may find in scientific literature.

Answer 28

• Selection bias: selection of individuals or groups in a study differs systematically from the population of interest
• Attrition bias: participants leave during a study
• Reporting bias : selective revealing or suppression of information by subjects
• Publication bias: outcome of study influences the decision whether to publish

Question 29

Give examples of possible inclusion/exclusion criteria.

Answer 29

• Study design
• Participants
• Intervention
• Outcome measure (primary)
• Outcome measure (secondary)

Question 30

How can trial identification be performed in a systematic review ?

Answer 30

• Electronic databases
• Identified trials by hand by searching the reference lists of identified studies and by consulting experts
• English and foreign language trials can be included

The way trial identification was performed has to be included in the systematic review

Question 31

What are the main pros and cons of systematic reviews ?

Answer 31

PROS

• Considered evidence based resource and the best guide to practice
• Results can be combined and statistically analysed as if they were one study
• Less costly to review studies than initiate a new one
• Comprehensive analysis of the best primary evidence using explicit and reproducible methodology

CONS

• Very time consuming
• Results of different trials often disagree
• Possible publication bias

Question 32

Describe the main features of meta analyses.

Answer 32

Combine qualitative and quantitative study data from several selected studies to develop a single conclusion that has greater statistical power.

1. Establish statistical significance with studies that have conflicting results
2. Develop a more accurate estimate of effect magnitude
3. Provide a more complex analysis of harms, safety data, and benefits
4. Examine subgroups with individual numbers that are not statistically significant

Question 33

What is the main difference between meta analyses and systematic reviews ?

Answer 33

“A systematic review answers a defined research question by collecting and summarising all empirical evidence that fits pre-specified eligibility criteria. A meta-analysis is the use of statistical methods to summarise the results of these studies.”

Question 34

What are the main pros and cons of meta-analyses ?

Answer 34

PROS

• Considered evidence based resource
• Greater statistical power
• Greater ability to extrapolate to the general population (because based on higher number of subjects)

CONS

• Very time consuming
• Requires advanced statistical techniques
• Study populations heterogeneity
• Results often disagree

Question 35

Give an example of problem of heterogeneity of study populations in meta analyses.

Answer 35

One study includes high number of people from Han Chinese population while other two studies did not.
Han population react badly to Allopurinol so if study was to look at effectiveness of that treatment, looking at studies together,
effects of that particular treatment in that population can be masked.