Critical Appraisal

Question 1

Define Validity

Answer 1

Is the finding true

Question 2

Define Generalisability

Answer 2

Is the finding applicable elsewhere?

Question 3

3 reasons for not being valid

Answer 3

Chance (random error)
—- (finding imprecise)
Bias (systematic error)
—- (finding inaccurate) 
Confounding (error of interpretation)
—- (beware, unknown confounders cannot be statistically accounted for)

Question 4

Define
Efficacy
Vs
Effectiveness

Answer 4

Efficacy: Shows if something works under ideal conditions
Vs
Effectiveness: shows if something works under normal conditions

Question 5

Types of research Qs:
Pragmatic research
Vs
Explanatory research

Answer 5

Pragmatic research
- demonstrates effectiveness 
Vs 
Explanatory research 
- demonstrated efficacy

Question 6

Aspects of

Pragmatic research

Answer 6

Unselected population
Pt centred outcomes
Does not interfere with clinical practice

Question 7

Aspects of

Explanatory research

Answer 7

Can it work under ideal conditions
Specific staff/setting/population
Often clinical outcome

Question 8

Null hypothesis

Define

Answer 8

No difference between active treatment and placebo

Question 9

P value

Define

Answer 9

Chance null hypothesis is true

Question 10

Type 1 error =

Answer 10

False positive (p value)

Question 11

Type 2 error =

Answer 11

False negative

Question 12

Statistical significance =
alpha + beta

Define a+b

Answer 12

Alpha
- maximum possibility of a false positive (p value)

Beta
- max possibility of false negative
(Determined by sample size)

Question 13

Power of study =

Answer 13

1- beta

Beta = max possibility of a false negative
Usually set to 80-90%

Question 14

Study power is set by

Answer 14

• Level of alpha
• Sample size
• The minimum clinical difference you wish to detect

Question 15

Issue with multiple hypothesis testing:

Answer 15

Increases false positive

- only use if there is a clear rationale to the test

Question 16

Benefit of randomisation

Answer 16

Removes confounders

Question 17

Concealment prevents

Answer 17

Allocation bias

Question 18

Blinding prevents

Answer 18

Measurement bias

Question 19

Outcome bias

More likely if

Answer 19

Soft outcomes

Question 20

Intention to treat

Answer 20

All analysis

Protects against bias

Question 21

Loss to follow up

Assume

Answer 21

Patients list are similar to analysed group.
Unless >30% LTFU
Or difference between test and control group

Question 22

Absolute risk =

Answer 22

Intervention - control

Question 23

Relative risk =

Answer 23

Intervention-control / proportion

Question 24

NNT=

Answer 24

1 / absolute risk

Question 25

Cluster randomisation

Issues

Answer 25

Allocation of concealment not possible

Therefore bias introduced

Question 26

Hawthorne effect =

Answer 26

Change due to being monitored

Question 27

Sustainability

Answer 27

What additional resources are required

What negative effects could occur due to intervention being rolled out

Question 28

Generalisability

Answer 28

X

Question 29

Independence of reference standard

Answer 29

X

Question 30

Work up bias

Answer 30

When a different reference standard is used

Question 31

Incorporation bias =

Answer 31

Diagnostic test forms part of reference standard

Question 32

Reliability =

Answer 32

Intraobserver error

Question 33

Effect of prevalence on
Sensitivity and specificity
PPV
NPV

Answer 33

Sensitivity and specificity
— remains constant 
PPV 
-increases with increasing prevalence
NPV
- decreases with increasing prevalence

Question 34

Likelihood ratio =

Answer 34

Value of information added
I =no value
>10 is diagnostic
<0.1 is rule out

Question 35

Forest plot =

Answer 35

Graph for heterogeneity

In a meta analysis

Question 36

Pros and cons

Meta analysis

Answer 36

Increased precision but same accuracy

Does not overcome bias of original data

Question 37

Areas of evaluation

For critical appraisal

Answer 37

Therapy
Service organisation
Diagnostic test
Systematic reviews

Question 38

Write an abstract

OD PICO RACS

Answer 38

Objective
Design

Population
Intervention
Comparator
Outcome

Results
Adverse events
Conclusions
Studies (further)

Question 39

Two main questions that critical appraisal tries to answer

Answer 39

1) is the result valid

2) is it generalisable

Question 40

Probability estimates of random error include

Answer 40

P value

Confidence intervals

Question 41

Main factor to consider in non randomised data

Answer 41

Confounders

Question 42

Inaccurate
Vs
Imprecise
Estimates

Answer 42

Inaccurate - due to bias
Vs
Imprecise - random error

Question 43

Confidence intervals and
Precision

Wide
Vs
Narrow

Answer 43

Wide - imprecise
Vs
Narrow - precise

Question 44

Statistics used to estimate effect of random error on results
2 types

Answer 44

P value
- for hypothesis testing

Confidence intervals
- for estimation of differences between groups

Question 45

P values and alpha values

And associations with false/true positive/negative

Answer 45

P value = probability of false positive (type 1 error)

Alpha value = probability of a true positive
(Max probability of false positive)

Beta value = probability of false negative

Question 46

Allocation or selection bias
How it occurs
And prevention

Answer 46

If patients or researchers can choose which treatment they get

Prevent via randomisation and concealment of allocation

Question 47

How to randomise service organisation as a intervention

Answer 47

Cluster randomisation

Disadvantage - no concealment if allocation

Question 48

80/20 rule

As a chart

Answer 48

Pareto chart
Bar chart with
cumulative line as percentage

Question 49

Likelihood ratio

Answer 49

The Likelihood Ratio (LR) is the likelihood that a given test result would be expected in a patient with the target disorder compared to the likelihood that that same result would be expected in a patient without the target disorder.