7.2 Review Of Evidence Based Medicine Flashcards

1
Q

What are the issues of p-values?

A
  • can never rule out chance
  • give no indication as to the size of the effect, just that the effect is statistically significant.
  • give no idea as to the range of uncertainty around the effect you have estimated
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is a confidence interval?

A

A 95% confidence interval captures uncertainty around the best estimate of relative risk. In a 95% confidence interval, there is a 95% probability that the true relative risk is within the interval.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is the line of no effect?

A

When relative risk ratio is equal to 1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

When are results said to be not statistically significant when looking at relevant risk?

A

When the confidence interval includes 1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

When are risk ratios and mean difference used?

A

Risk ratios are used if you are using a binary outcome.

Mean difference is used if you care looking at continuous outcome.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

what is the risk ratio?

A

ratio of incidence, comparing multiple groups. incidence rate in group A vs. incidence rate in group B

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

what is the odds ratio?

A

(used most often in case/control studies) = ratio of odds of ‘outcome’ in exposed group vs. odds of ‘outcome’ in unexposed group

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

what is the absolute risk?

A

the risk of acquiring a

given disease over a given period of time.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

what is the absolute risk difference?

A

control event rate - intervention event rate

or unexposed event rate – exposed event rate

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

what is the p-value?

A

an expression of statistical significance. It is the probability that the effect observes could have occurred by chance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

what does a small p-value imply?

A

that there is a small chance of the measured effect not being a real effect of a given drug. effect is unlikely to be down to chance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

what does a large p-value imply?

A

that there is a greater probability of the observed effect being down to chance rather than the real effect of the drug

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

when do we say that an observation is considered statistically significant and not due to chance?

A

Traditionally, p-values <0.05 are considered ‘statistically
significant’, i.e. we are ‘happy’ to discount a 5% chance
effect

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

when do confidence intervals indicate that the study findings are not statistically significant?

A

when the confidence interval for relative risk spans 1 (the line of no effect)

when the confidence interval spans 0 for mean absolute difference (no difference)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

if the relative risk is negative what does this indicate?

A

that the risk for the control is less - favours control

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

if the relative risk is above 1, what does this mean?

A

that the risk for the intervention is less than that of the control - favours intervention

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

what does MAD stand for?

A

mean absolute difference

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

what is the mean absolute difference value of no effect?

A

0

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

what are the 2 broad categories of quantitative study design?

A

experimental

observational

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

what are the different types of observational studies?

A
cross-sectional
case-control
case studies
case series
cohort studies
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

what are the different types of experimental studies?

A

RCTs

other experimental non-randomised designs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

what is the hierarchy of evidence from top to bottom?

A
  1. systematic review of randomised trials
  2. randomised controlled trial
  3. controlled, non-randomised study
  4. observational studies (cohort, case control, cross sectional)
  5. case series
  6. case study
  7. consensus / expert views
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

where can you look to conduct a high quality literature search?

A
BMJ best practice 
NICE guidance
Cochrane library 
Medline
PubMed
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

what is the PICOS technique?

A

a mechanism to searching for literature.
population or patient group
interventions / investigations considered
comparator / control
outcomes considered
study design

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

what is a confounder?

A

A confounder is a factor associated with the exposure and is independently a risk factor for the disease.

26
Q

what are some biases we need to consider in studies?

A

selection bias
recall or reporting bias
observer bias

27
Q

give some examples of when its hard to do blinding

A

Surgical procedures Psychotherapy vs. anti-depressant
Alternative medicine, e.g. acupuncture, vs. Western medicine, e.g. drug
Lifestyle interventions
Prevention programmes

28
Q

what is an explanatory trial?

A

an as-treated analysis
Analyses only those who completed follow- up and complied with treatments
Compares the physiological effects of the treatments, but doesn’t consider the adherence

29
Q

what is the disadvantages of explanatory trials?

A

it loses the effects of randomisation as non-compliers are likely to be systematically different from the compliers resulting in selection bias and confounding

30
Q

what is a pragmatic trial?

A

an intention to treat analysis. analyses according to the original allocation to treatment groups (regardless of whether they completed follow up or adhered to treatment)
compares the likely effects of using the treatments in routine clinical practice

31
Q

what are the benefits of using a pragmatic trial over a explanatory trial?

A

it preserves the effects of randomisation and minimises selection bias and confounding factors

32
Q

what are the 2 different forms of literature reviews of studies

A

narrative reviews

systemic reviews

33
Q

what are the disadvantages of a narrative review?

A

implicit assumptions, opaque

methodology, not reproducible ⇒ biased, subjective

34
Q

what are the advantages of systematic reviews?

A

explicit assumptions, transparent methodology, reproducible ⇒ unbiased, objective

35
Q

what is considered in the decision analysis?

A

harm and benefits

cost-effectiveness

36
Q

what is a systematic review?

A

an overview of primary studies that used explicit and reproducible methods”
must be explicit, transparent and reproducible

37
Q

what is a meta-analysis?

A

a quantitative synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way”

38
Q

what is the purpose of a meta-analysis?

A
  • To facilitate the synthesis of a large number of study results
  • To systematically collate study results
  • To reduce problems of interpretation due to variations in sampling
  • To quantify effect sizes and their uncertainty as a pooled estimate
39
Q

what should the formal protocol of a meta-analysis specify?

A
  • compilation of complete set of studies
  • identification of common variable or category definition
  • standardised data extraction • analysis allowing for sources of variation
40
Q

how are studies weighted in a meta-analysis?

A

according to their size and the uncertainty of their odds ratio (narrower 95% confidence interval means that there is a greater weighting to the result)

41
Q

what does the diamond on a forest plot represent?

A

the meta analysis estimate, the width of the diamond represents the upper and lower limits of the confidence interval

42
Q

how do we interpret forest plots?

A

individual odds ratios (squares) with their 95% CI (lines) are displayed for each study
size of the square is proportional to the weight given to the study
the diamond is the pooled estimate with the centre indicating the pooled odds ratio (dotted line) and the width representing the 95% CI
the solid line is the null hypothesis

43
Q

what are the key features of a systematic review?

A

– formal protocol

– explicit, transparent and reproducible

44
Q

what are the key features of a meta-analysis?

A

– a quantitative synthesis of primary data
– summarises effect sizes and their uncertainty
– displayed as a Forest Plot

45
Q

what are the problems with meta-analysis?

A

• Heterogeneity between studies:
– Modelling for variation (Fixed effect model vs. Random effects model)
– Analysing the variation (Sub-group analysis)
• Variable quality of the studies
• Publication bias in selection of studies

46
Q

what is the fixed effect model

A

a model for approaching the calculation of the pooled estimate odds ratio and its 95% CI in a meta-analysis
Assumes that the studies
are estimating exactly the same true effect size, and that the only variation between the effect estimate in individual studies is due to random variation

47
Q

what is the random effects model?

A

A model for approaching the calculation of the pooled estimate odds ratio and its 95% CI in a meta-analysis
It assumes that the studies are estimating similar, but not the same true effect size

48
Q

describe the structure of a fixed effects model graph

A

the central line is the true effect line. There is only one effect line as all of the studies are considered to be estimating exactly the same true effect size.
The line placement of true effect is placed in the area that minimises the distance from the line to the individual study dots. Studies are weighted for the uncertainty.
the distance between the study result dots and the true effect line is considered to be due to random error

49
Q

describe the structure of a random effects model?

A

there is a line of true mean effect that is placed in the mean location based on the true trial specific effect on the different studies.
Each study is measuring a different effect and therefore produces a different line, this considers clinical heterogeneity.

50
Q

why is the confidence interval in the random effects model wider than in a fixed effects model?

A

as smaller studies are up weighted in a random effects model as every model is measuring a different treatment effect

51
Q

if the study variance is low, what does this imply?

A

that the heterogeneity between studies is low.

52
Q

how do we analyse the heterogeneity between studies/variation?

A

Sub-group analysis can help to explain heterogeneity which may provide further
insight into the effect of a treatment or exposure
– Study characteristics (e.g. year of publication, length of follow up, %female participants)
– Participant profile – where data is analysed by types of participants (e.g. subgroups of males, females, adults, children)

53
Q

what can cause variation in the quality of studies?

A

– poor study design
– poor design protocol
– poor protocol implementation

54
Q

what 2 approaches are used to overcome the variation in the quality of studies when doing a meta-analysis?

A

Two approaches tend to be used:
1. define a basic quality standard and only
include studies satisfying this criteria, e.g. Cochrane reviews used to include only RCTs

  1. score each study for its quality and then incorporate the quality score into the weighting allocated to each study during the modelling, so that higher quality studies have a greater influence on the pooled estimate – use sub-group analyses to explore differences, e.g. high quality studies vs. low quality studies
55
Q

what components are considered when assessing the quality of a randomised control trial?

A

– allocation methods, e.g. randomisation?
– blinding and outcome assessment
– patient attrition, e.g. <10% & Intention-to-Treat (ITT)
– appropriate statistical analysis

56
Q

what is publication bias?

A

Studies with statistically significant or ‘favourable’ results are more likely to be published than those studies with non- statistically significant or ‘unfavourable’ results – this applies particularly to smaller studies
Publication bias leads to a biased selection of studies towards demonstration of effect

57
Q

what can we use to identify publication bias in selection studies?

A

– check protocol for search strategy
– look at a Funnel Plot
– use a statistical test

58
Q

what are funnel plots used for?

A

to identify publication bias in selection of studies.

59
Q

what are funnel plots?

A

A plot of some measure of study size (e.g. standard error of estimate) against measure of effect (e.g. odds ratio)
If no publication bias, then the plot will be a ‘balanced’/ symmetrical funnel
Smaller studies can be expected to vary further
from the ‘central’ effect size
Publication bias is likely to exist if there are few small studies with results indicating small or ‘negative’ measure of effect

60
Q

what are sources of evidence for systematic reviews?

A

Cochrane library
NHS centre for reviews and dissemination
NIHR health technology assessment programme