Statistics and Critical Appraisal

Question 1

what is a proportion?

Answer 1

number of events occuring/number in group

Question 2

what is risk/prevalence (P)?

Answer 2

the prevalence/risk of an event occurring

P event=no. people experiencing event/total no. people in group

Question 3

what is relative risk?

Answer 3

how much more likely the event is to occur in one group compared to another

Question 4

how do you calculate relative risk?

Answer 4

exposed group/risk of event in unexposed group

Question 5

what does a risk ratio >1 indicate?

Answer 5

the risk of disease for the exposed group is larger than the risk of disease for the unexposed group

Question 6

what other statistical measure is usually reported alongside risk?

Answer 6

confidence interval

Question 7

what is relative risk difference/attributable risk?

Answer 7

1 - relative risk (when RR<1)

Question 8

how do you calculate the absolute risk difference?

Answer 8

risk in exposed - risk in unexposed

Question 9

what is the number needed to treat?

Answer 9

the additional number of people you would need to give a new treatment to in order to cure one extra person compared to old treatment

Question 10

how is number needed to treat calculated?

Answer 10

calculated using absolute risk difference

=1/ARD

Question 11

how do you calculate odds?

Answer 11

p/(1-p)

p=probability of event occurring

Question 12

how do you calculate odds ratio?

Answer 12

odds exposed group/odds unexposed group

Question 13

what does it indicate if the odds ratio>1?

Answer 13

the odds of disease in the exposed group are larger than the odds of disease in the unexposed group, exposure to the factor increases the risk of contracting the disease

Question 14

when will the odds ratio and relative risk be similar?

Answer 14

total sample is large and event is rare

Question 15

what is absolute risk?

Answer 15

measure of risk of a certain event happening

more useful at communicating true impact

Question 16

what is relative risk?

Answer 16

measure of the risk of a certain event happening in one group compared with the risk of the same event happening in another group (RR of 1 means there is no difference between groups

Question 17

what type of study is best placed to answer questions about effectiveness of a particular therapy?

Answer 17

RCT

systemic reviews of RCT

Question 18

What are the different kinds of RCT?

Answer 18

factorial trials

cluster trials

Question 19

what is a factorial trial?

Answer 19

several therapies may be evaluated at same time by allocating pts to combinations of drugs

Question 20

what is a cluster trial?

Answer 20

groups of patients are randomised (rather than individual patients being assigned to a particular Tx) i.e. all pts at one surgery/ on one ward

Question 21

What type(s) of study design are best placed to answer questions about the aetiology of / risk factors for a condition?

Answer 21

cohort or Case-Control study

Question 22

in what situation is a case control the only option?

Answer 22

If the condition under study is rare, or the time from exposure to development of the condition is particularly long, a case-control study may be the only realistic possibility, despite the drawbacks of case-control study methodology

Question 23

What type(s) of study would be best suited to answering questions about the prognosis of a particular condition?

Answer 23

cohort

Question 24

what questions are asked in a critical appraisal?

Answer 24

Are results of study valid?
What are results? Are they important?
Are results applicable to our patient/situation?

Question 25

describe a randomised control trial?

Answer 25

qRandomised controlled trial recruits eligible participants then randomly allocates them to a treatment arm. It is very important that the allocation is done in a truly random manner, for example by computer-generated random numbers. There are other types of trial that are not randomised. It is preferable that wherever possible the treatment allocated is kept secret from both the participants and those assessing the results – this is called double blinding

Question 26

Why is randomization the best allocation method for interventions?

Answer 26

1. Randomisation + allocation concealment used to minimize selection bias, to ensure Tx and control groups are comparable (groups similar across all patient characteristics and without significant differences at baseline).
2. This allows us to correctly identify + quantify Tx effects (or lack thereof)
3. Incomplete randomization leads to selection bias, where there are significant baseline differences between the 2 groups.
4. Any differences between groups at baseline leads us to over- or under- estimating the effect of the treatment (usually leads to over-estimation of Tx effect)

Question 27

give an example of good randomisation?

Answer 27

truly random. E.g. done by computer-generated random numbers

Question 28

give an example of an inadequate method of randomisation?

Answer 28

randomization based on date of presentation, order of presentation to clinician, letter of surname/number in date of birth etc.
These are inadequate methods as they may allow recruiting clinician to predict which treatment group the patients might be in, before deciding to recruit them into the study- leading to baseline differences between study groups

Question 29

what are the consequences if the intervention and control group are different at baseline in RCT?

Answer 29

Any differences between treatment + control groups at baseline leads to us over- or under-estimating the treatment effect. Usually it leads to overestimation of treatment effectiveness.
This means we conclude there’s a significant difference in outcome between the two groups, where in fact there is not, this is known as Type 1 statistical error.

Question 30

why is allocation concealment necessary in RCT?

Answer 30

Allocation concealment prevents clinicians performing the allocation from predicting what group patients would be in before deciding to recruit them to the trial.

Question 31

What are the 2 forms of bias that allocation concealment aims to prevent?

Answer 31

selection bias

subversion bias

Question 32

what is selection bias?

Answer 32

Bias in selection of study population

Question 33

what is subversion bias?

Answer 33

Deliberate manipulation of patient recruitment by clinicians such that groups are not equal at baseline, in order to influence trial results
• May be done actively e.g. shining light through envelope to determine pt’s allocation group
• Or more subtly e.g. when a clinician decides not to recruit a pt to a trial who they feel might not do well because they’re in the Tx group, or vice-versa

Question 34

what is double blinding?

Answer 34

o Means concealing the intervention from both the patient and the person assessing the treatment effect
o Blinding of patients- their beliefs about the treatment received may influence reporting of the effects and side-effects of treatment, and their decision to continue with the trial
o Blinding of clinician/assessor- knowledge of Tx delivered may influence their recording or interpretation of results, leading to detection bias
(If the person recording the outcome knows which group the pt was in, this can also be referred to as ascertainment bias or expectation bias)
Blinding is most important when aspects of outcome measurement are subjective

Question 35

what is confounding?

Answer 35

refers to presence of factors other than those being studied which are present unequally between the two study arms at baseline + influence the result, leading to the erroneous conclusion that the intervention being studied is responsible for different outcomes.

Question 36

how is confounding prevented in RCT?

Answer 36

Random allocation to different treatment arms in a RCT aims to reduce confounding, as any factors should be divided fairly equally.
To ‘help’ randomization, researchers may have to ‘stratify’ randomization
This means you know of a very powerful confounder (e.g. age- you’d need to split participants into ‘young’ and ‘old’ before you randomize; in order to ensure you have same proportion of young + old in each group)

Question 37

what is equipoise?

Answer 37

situation where researchers have no preference between treatments being studied in a trial. Can be collective (where profession as a whole has no preference between treaments) or individual (referring to the individual clinician recruiting patients into trials)

Question 38

what are the problems when equipoise is lost?

Answer 38

when those designing trial or recruiting patients into it do have a preference for one treatment over another. It can be considered unethical to recruit patients into the trial as you’re offering some patients what you believe to be a “worse option”.

Question 39

how do you decide how many people need to be in a trial?

Answer 39

o This refers to power calculation
o Power = likelihood of detecting a true difference if it’s present and is expressed as a percentage. 80% or 90% power levels = acceptable

Step 1- decide what would constitute a clinically significant effect (not just statistically significant)

Step 2- know the underlying variability of the principle outcome variable, and you need to decide what statistical significance level will be used (p=0.05 normally)

Step 3- calculate a minimum sample size necessary to identidy the effect you want to identify, if it exists. Establish from pre-existing tables.

Question 40

what are the problems with an under-powered trial?

Answer 40

1. Has insufficient participants to correctly distinguish the real effect (or lack of effect) of a treatment from random variation.
2. May lead to erroneously concluding that a treatment has no effect, when in fact it does- Type 2 statistical error

Question 41

what is a cohort study?

Answer 41

Cohort= Observational study- no intervention by researchers. Identifies a cohort of patients and follows them up over a period of time, measuring exposures and outcomes of interest.
A variant is a “historical cohort study”, when a cohort of patients is identified “in the past” and outcomes + exposures are then measured retrospectively up to the present

Question 42

what are confounding factors?

Answer 42

Factors (separate from ones being studied) that influence the outcome being studied, and which are present unequally between the two groups, meaning that the groups are significantly different at baseline

Question 43

why is it important to anticipate confounding?

Answer 43

o Two groups in a cohort (or case-control) study aren’t selected at random. They differ in an important way (the intervention, exposure or outcome being studied), therefore they may differ in other important ways at baseline
o Because of this non-random group selection, observational studies are particularly prone to selection bias

Question 44

What can go wrong if we don’t accurately characterise exposure when undertaking a cohort study?

Answer 44

non differential error

differential error

Question 45

what is non-differential error?

Answer 45

Would occur if likelihood that exposure wasn’t accurately measured was same for different exposure categories
E.g. Measuring BP as an exposure- not perfectly accurate- there will be some classed as ‘high blood pressure’ at the time who are really ‘low’, and vice-versa. This misclassification probably balances itself out- considered ‘non-differential’
Implication- we end up comparing outcomes in groups which overlap a bit- so we get a biased estimate of how much difference there is in these outcomes- tends towards ‘null’ (no-difference)

Question 46

what is differential error?

Answer 46

Occurs when exposure is less accurate in specific categories
E.g. Self reported BMI- we know that those with high BMI tend to under-report systematically, and this doesn’t balance out- so is differential error.
Direction of bias is unpredictable, can lead to over- or under-estimate of true effects of exposure

Question 47

What goes wrong if we don’t have same level of surveillance for outcomes to all individuals in the cohort?

Answer 47

o How hard we look influences what we find- look harder and you’ll find more outcomes. If you look harder in one group than another, you’ll see artefactual differences between groups
o Ideally want to see same level of surveillance (no of follow-up questionnaires, same level of completeness etc.)

Question 48

What are the 2 types of outcomes in a cohort study?

Answer 48

o Objective outcomes, e.g. death, are clear and easy to quantify
o Subjective outcomes (patients perspective or investigator’s assessment) need a validated tool to measure outcomes, to minimize bias.

Question 49

describe blinding in cohort studies?

Answer 49

Needed because if study analysts know which group a patient is in, it may make them more likely to search harder for the event of interest and/or recording more outcomes for patients in the exposure group
=> leads to bias + overestimating effect of exposure on the outcome

Question 50

how can researchers address confounding?

Answer 50

o Systematic approach to IDing potential confounders- you can only address the factors you know about (may be unknown confounders)
o Identify => Methodological/Statistical techniques used to eliminate confounders, or to assess their potential effect on the level of difference between patient groups at baseline; and to adjust for their effect on the study results

Techniques:
 Restriction- limiting study participants to those w/o confounding factor
 Stratification- analyzing results in subgroups according to potential confounders
 Multi-variable regression

Question 51

what is a case control study?

Answer 51

Observation study- implies no active study intervention.
Case control- investigators select “cases” (individuals/ patients with the disease or outcome of note) and matches them to “controls” (individuals / patients who are similar, except that they do not have the disease or outcome). It then compares their exposure to the risk factor or intervention being studied. This type of study does not involve the long periods of follow-up that a cohort study does, and if a rare outcome is being studied it may be the only realistic way of answering questions about risk factor

Question 52

what are the 3 discrete steps of appraisal?

Answer 52

• are the results valid (bias
• what are the results
• are the results applicable to your question and patient care

Question 53

what results are shown in a RCT?

Answer 53

look @ Relative/absolute risk change, odds ratio, NNT, confidence intervals

Question 54

what results are shown in diagnosis studies?

Answer 54

look @ sensitivity, specificity, +ve/-ve predictive value, likelihood ratios

Question 55

what results are shown in prognosis studies?

Answer 55

Relative risks/odds ratios – how likely event is over time in the pt, how precise this is.

Question 56

what results are given in a harm/aetiology study?

Answer 56

Look @ relative risk, odds ratio, NNH (numbers needed to harm)

Question 57

what results are shown in forrest plots?

Answer 57

Look at Heterogeneity

if all different then meta-analysis is MEANINGLESS!

Question 58

what do you look for when assessing if a study is applicable to your question and patient care?

Answer 58

• Generalisability/Particularisability – how similar are study pop. to the pt?
• Does local health care provide test/intervention? Possible to do this where you are?
• Benefits/harms? Opportunity cost?