Critical appraisal

Question 1

Randomisation

Answer 1

Purpose is to try and ensure that any characteristics of the sample population that may affect the results ( confounders) are distributed equally between the two study groups, and avoiding selection bias

Question 2

Tools for randomisation:

Answer 2

centralized computer randomization (with contact by phone or computer) is ideal and often used in multicentre trials.
smaller trials may use an independent person (eg the hospital pharmacy to manage the randomization

Question 3

Stratified randomisation

Answer 3

With powerful confounders ( eg age, sex) , patients can first be split, or stratified , into different groups before randomisation, so there will be the same number of patients with and without the confounder ( young and old, male and female) in each arm of the trial

Question 4

allocation concealment

Answer 4

whether the person recruiting the patient to the trial could know or anticipate the group allocation that patient would receive, preventing selection bias
Prevents clinicians predicting which group patients would be in before recruiting them to the trial

Question 5

Blinding

Answer 5

When some of the partiticapits of a trial ( patient/clinicians/researchers) are prevented from knowing certain information that may lead to conscious or subconscious bias

Question 6

Blinding advantages

Answer 6

• Prevents observer bias = form of reactivity in which researcher’s cognitive bias causes them to subconsciously influence the participants of experiment = could influence extra quality of care to these patients
• Or confirmation bias = see results that aren’t there
• Expectation bias (Pygmalion effect) = Observers may subconsciously measure or report data in a way that favours the expected study outcome
  Also prevents placebo effect – or reduces it

Question 7

double blinded trial

Answer 7

both patients and investigators are unaware of treatment allocation

Question 8

when blinding is best to use

Answer 8

outcome is subjective (eg measurement of symptoms or function)
or if outcome measurement is based on patient self report

Question 9

instances when blinding is not needed

Answer 9

if outcome is objective eg death
sometimes impossible to achieve eg in a trial involving physiotherapy , they will know whether or not they have received it
in a trial involving warfarin ,clinical cannot be blinded due to safety reasons

Question 10

Confounding

Answer 10

distortion (or potential for distortion) of association between outcome and exposure
by third factor
which has an association with both exposure and outcome

Question 11

common causes of confounding.

Answer 11

Confounding occurs when there is a non random distribution of risk factors in the populations. Age, sex and social class are common causes

Question 12

How to control confounding factors

Answer 12

In the design stage of an experiment, confounding can be controlled by randomisation which aims to produce an even amount of potential risk factors in two populations.
In the analysis stage of an experiment, confounding can be controlled for by stratification.

Question 13

Study least subject to bias

Answer 13

RCT - the groups are likely to be similar with respect to known and unknown determinants of outcome therefore we can be more confident that any observed differences in outcome are due to the intervention.

Question 14

‘intention to treat’ analysis

Answer 14

Statistical analysis of data from subjects according to the group to which they were assigned despite noncompliance with the study protocol

Question 15

‘per protocol’ analysis

Answer 15

An analysis of patient outcomes based only on those subjects who completed all aspects of the protocol. Also called on-treatment analysis.

Question 16

Treatment fidelity

Answer 16

how accurately the intervention is reproduced from a protocol or model

Question 17

Validity

Answer 17

describes how accurately a study, instrument, test or equivalent measures what it is supposed to.

Question 18

Factors affecting validity

Answer 18

study size
inter-participant variability the use of different measuring instruments (instrumentality)
Certain biases such as attrition and selection bias

Question 19

internal validity

Answer 19

how well the study was conducted, the degree to which the effects observed in an experiment are due to the independent variable and not confounds-true, accuracy

Question 20

Threats to internal validity

Answer 20

Reliability of measurement instruments
Regression towards the mean
Sampling
Experimental mortality
Instrument obtrusiveness
Maturation
Measurement instrument learning

Question 21

external validity

Answer 21

extent to which we can generalise findings to real-world settings-useful, generalisability

Question 22

Threats to external validity:

Answer 22

Representativeness of the sample
Reactive effects of setting (is the research setting artificial)
Effect of testing (if a pre-test was used in the study that will not be used in the real world this may affect outcomes)
Multiple treatment inference (this refers to study’s in which subject receive more than one treatment, the effects of multiple treatments may interact)

Question 23

Reliability

Answer 23

is the extent to which an experiment, test, or any measuring procedure yields the same result on repeated trials.The higher the reliability the more likely you are to obtain similar results if the study was repeated.Reliability does not ensure accuracy.

Question 24

Cohort study

Answer 24

sample that has been exposed to a certain exposure and follow that sample to observe the outcome. Cohort studies can be retrospective or prospective. used for prognosis and studying rare exposures. But if uncommon event cohort study would have to be unfeasibly large to answer the study question

Question 25

Cohort study advantages

Answer 25

- Best information about causation of a disease, can work out incidence - Able to examine a range of outcomes from a particular exposure - good for rare exposures

Question 26

Cohort studies negatives

Answer 26

- Often large, difficult to follow up large groups of patients, especially with something such as monitoring diet, expensive and time-consuming - Hard to conduct if length of time from exposure to outcome is very long (eg for some cancers) or if exposure you're observing is rare - Need to look out for confounders - Bad for long latent periods - bad for rare outcomes - Misclassified exposure - Different follow up for exposed/ non exposed - Outcomes assessors not blinded to exposure category - Selection bias eg sample patients all live near a nuclear power plant

Question 27

Loss of follow up can result in :

Answer 27

- measurement error reduce available sample size and effect study's ability to detect a true association between exposure and outcome->increases chance of type 2 statistical error if it is systematic, it will introduce bias

Question 28

Limiting confounding in a cohort study

Answer 28

Restriction - limit participants of study that have possible confounders Matching and stratification - make comparison groups,adjust for confounding Multiple variable regression - coefficients are established for the confounder groups. Allows for better adjustment

Question 29

Bradford hill's criteria

Answer 29

* Strength of association * Specificity - Does A always only cause B? * Temporal association - effect has to come after cause * Theoretical plausibility * Consistency - Do you always find the same relationship? * Coherence - Does the data fit in with what we know now? * Dose-response relationship - Does greater exposure lead to greater effect? * Experimental evidence - Can we test this experimentally? * Analogy - If A causes B, does something similar to A cause something similar to B?

Question 30

Case control study

Answer 30

sample that already has a certain outcome, follow them back to find out if they were exposed to a certain exposure.Case-control studies are retrospective. They clearly define two groups at the start: one with the outcome/disease and one without the outcome/disease. They look back to assess whether there is a statistically significant difference in the rates of exposure to a defined risk factor between the groups.used for Studying cause of rare diseases

Question 31

Challenges to measuring exposures in case controls:

Answer 31

Recall bias Variable exposure- patients environment may have changed eg moved house Unavailable data - eg patient can't remember, medical records unavailable, incomplete or inaccurate

Question 32

recall bias

Answer 32

participant cannot remember when they were exposed, or their outcome changes their perception of the exposure

Question 33

Case-control studies negatives

Answer 33

- Often affected by recall bias participant cannot remember when they were exposed, or their outcome changes their perception of the exposure - Or affected by selection bias where control group has other factors that may influence their exposure - Needs a large sample size for rare exposures - Cases don't represent the full disease spectrum - Confounders need to be recognised/addressed

Question 34

Case control advantages

Answer 34

- simple/easy to conduct, do not require long follow up, outcome already present - Good for rare outcomes, can select all patients with a certain disease - Good for long latent periods, not waiting for it long after the exposure

Question 35

Observational study bias

Answer 35

In an observational study patient or clinician preference rather than randomisation determines whether a patient is allocated to the intervention or comparison group. In the absence of randomisation there is a greater risk of imbalance in both the known and unknown determinants of outcome, and consequently any observed differences in outcome might be unrelated to the intervention but due to differences between groups at baseline

Question 36

systemtic reviews

Answer 36

A rigorous summary of all the research evidence that relates to a specific question

Question 37

Finding all the evidence in a systematic review:

Answer 37

Searching bibliographic databases eg medline Searching for non english papers( avoids language bias) Searching through references of other trials Trial registries Contacting authors Conducting a thorough search for evidence in a systematic review can eliminate publication bias, language bias and selection bias

Question 38

Systematic review- how to appraise validity

Answer 38

Is this a systematic review of RCTs? - anything less than RCT is inadequate What was the search strategy? Studies with negative results or foreign languages are unlikely to be included How was validity of individual studies assessed? Are the results consistent from study to study?

Question 39

disadvantages of single studies:

Answer 39

Individual studies may have inadequate sample sizes to detect important differences (leading to false negative results).The results of apparently similar studies may vary because of chance. Subtle differences in the design of studies and the participants may lead to different or even discrepant findings).

Question 40

A systematic review has some advantages over a primary data study:

Answer 40

first, speed (why conduct a new RCT if the answer is already available in published research?); second, ethics (is it defensible to conduct a controlled trial if the answer is available in trials already conducted?); third, statistical power (all else being equal, combining data from more than one study increases certainty in findings and precision of estimates). By bringing together all the relevant evidence, disadvantages of single studies can be guarded against

Question 41

Sensitivity

Answer 41

Proportion of patients with the disease who get a positive = No. True positives / all those with disease Tests with high sensitivity correctly classify a high proportion of people who really have disease

Question 42

Specificity

Answer 42

Proportion of patients without the disease who get a negative test = No. Of true negatives / all those without disease Tests with high specificity correctly classify a high proportion of people who don’t have disease

Question 43

Positive predictive value

Answer 43

Chance of having disease if your test is positive No. Of true positives/ all those that test positive As prevalence Rises this also rises

Question 44

Negative predictive value

Answer 44

Chance of not having disease if your test is negative No. Of true negatives / all those test negative As prevalence increases this value falls

Question 45

Likelihood ratio for positive test result

Answer 45

•Sensitivity ÷ (1 – Specificity)

Question 46

Likelihood ratio for negative test result

Answer 46

(1 – Sensitivity) ÷ Specificity

Question 47

Risk

Answer 47

a ratio of the number of people who develop an outcome to the total number of people

Question 48

Risk ratio

Answer 48

probability of disease/ risk in exposed/ probability of disease/risk in unexposed

Question 49

RR<1

Answer 49

means that the treatment decreases risk of the outcome, the rate of an event is decreased compared to controls. The relative risk reduction should therefore be calculated

Question 50

RR>1

Answer 50

means that the treatment increased the risk of the outcome. the rate of an event (eg experiencing significant pain relief) is increased compared to controls. It is therefore appropriate to calculate the relative risk increase if necessary

Question 51

RR 1

Answer 51

means no difference between the groups (treatment has no effect)

Question 52

Number needed to treat (NNT)

Answer 52

number needed to be treated to produce one improved outcome (1 ÷ ARR) Round up to a whole number

Question 53

Number needed to harm

Answer 53

number needed to be treated to produce one harmful outcome Round value down to a whole number can be calculated by dividing 1 by the absolute risk increase

Question 54

95% confidence interval

Answer 54

the range of values of the study sample within which we can be 95% sure the true population value lies

Question 55

statistically significant test result

Answer 55

(P ≤ 0.05) means that the test hypothesis is false or should be rejected.

Question 56

Type 1 error

Answer 56

``` (False positive) Rejecting null hypothesis when it is true shows a difference when there is none Poor study Data manipulation ```

Question 57

Type 2 error

Answer 57

(False negative) Accepting null hypothesis when you should have rejected it/the null hypothesis is false A type two error occurs when a study fails to detect an effect or an association that does exist. i.e. detects no difference when there is one Might be related to sample size

Question 58

Power

Answer 58

probability of correctly rejecting the null hypothesis when it is false = 1 - probability of type 2 error the chance of detecting a true difference when there really is one

Question 59

Power can be increased by

Answer 59

increasing sample size, using more precise measuring instruments, and using a higher significance value.

Question 60

Odds

Answer 60

a ratio of the number of people who develop an outcome to the number of people who don’t

Question 61

usually used in case control studies

Answer 61

odds ratio | Odds ratios are always bigger so they look better in a paper abstract. Odds can be >1 but risk is always <1

Question 62

Multivariate analysis

Answer 62

allow confounding factors to be taken into account, by adjusting for these factors.performed using statistical models.

Question 63

Hazard ratio

Answer 63

measure of an effect of an intervention on an outcome of interest over time.

Question 64

Precision

Answer 64

quantifies a test's ability to produce the same measurements with repeated tests.

Question 65

Bias

Answer 65

systematic introduction of error into a study that can distort the results in a non-random way is the tendency of a statistic to overestimate or underestimate a parameter.

Question 66

Selection bias

Answer 66

Error in assigning individuals to groups leading to differences that may influence the outcome. The subjects are not representative of the population Especially a problem in cohort studies

Question 67

Types of selection bias

Answer 67

1. sampling bias ( eg due to non random sample of population) 2. volunteer/non responder bias 3. Time interval bias 4. Attrition/ loss to follow up bias 5. prevalence/incidence bias (Neyman bias)- study is investigating a condition that is characterised by early fatalities or silent cases 6. admission bias (Berkson's bias) 7. healthy worker effect

Question 68

Recall bias

Answer 68

Difference in the accuracy of recollection of study participants, possible due to whether they have the outcome or not Especially a problem in case control studies

Question 69

Publication bias

Answer 69

Failure to publish/include results from valid studies, often because they show a negative or uninteresting result Important in meta-analyses and systematic reviews where studies showing negative results may be excluded

Question 70

Hawthorne effect

Answer 70

Group changing its behaviour due to the knowledge that it is being studied

Question 71

Procedure bias

Answer 71

Subjects in different groups receive different treatment, other than just the interventions Eg elderly patients could receive human contact along with intervention whereas control patients may get no extra human contact

Question 72

Measurement bias

Answer 72

It occurs when the accuracy of information collected about or from study participants is not equal between cases and controls Quantitive eg: due to poor calibration of measuring instruments Qualitative eg: study participant is less likely to answer a question due to stigma associated with the answer

Question 73

Lead time bias

Answer 73

Early diagnosis appears to prolong survival

Question 74

Length bias

Answer 74

Screening over represents less aggressive disease A reason why cancers detected by screening may on average be more slowly progressive. screening tends to detect disease which progresses more slowly and has long asymptomatic periods.. Conversely, aggressive disease is more likely to be missed by screening

Question 75

Effective Reproduction Number

Answer 75

is the average number of secondary infections produced by a typical infective agent; if this number is greater than 1 then it is impossible to eradicate an infection.

Question 76

Cost Effectiveness Analysis (CEA):

Answer 76

Used when the effect (outcome) of the two interventions is expected to vary The outcome is measured in natural units e.g. BP, cholesterol level, mortality, live years saved The outcome is one dimensional - addresses quantity or quality, not both Costs & outcomes are combined into a single measure to allow comparison

Question 77

Cost Utility Analysis (CUA):

Answer 77

Used when the effect (outcome) of the interventions on health status has two or more dimensions Measures outcome in terms of quantity and quality. Combines these into a single measure e.g. the QALY = Quality Adjusted Life Year A measure which tries to combine a quantitative measure (months gained, years gained) with a qualitative measure of the quality of that measure Can be used to compare interventions with a disease or condition or across different diseases or treatment options

Question 78

QALY

Answer 78

Based on number of years of life that would be added by intervention Each year of perfect health = 1, to a value of 0 for death If extra years are not lived in full health (eg loss of limb) = between 0 and 1

Question 79

Cost Benefit Analysis (CBA):

Answer 79

Places a monetary value on benefits or outcomes Generally based on individuals’ observed or stated preferences and values for something Most common approach is “willingness to pay” most comprehensive but rarely undertaken.

Question 80

Cost Minimisation Analysis (CMA):

Answer 80

Used when the effect (outcome) of both interventions is identical (or assumed to be identical) No outcome measurement Only costs are accounted for cannot provide answers where effectiveness is different between competing alternatives

Question 81

Cost Consequence:

Answer 81

Measures costs, measures consequences

Question 82

Sensitivity Analysis:

Answer 82

‘Vary key assumptions and see if it has an impact’

Question 83

Incremental cost effectiveness ratio

Answer 83

to work out cost effectiveness This is the ratio of the change in costs of therapeutic interventions (compared with alternatives/no treatment) to the change in effects of the intervention ( measured as clinical outcome of QALY) ICER=(Cost A- Cost B) / (QALYs b -QALYsA)

Question 84

Why - Combining data in a meta analysis

Answer 84

Increases the number of patients being analysed ( increases size of study sample) Improves precision and reduces the width of the confidence intervals Can demonstrate a statistically significant result when none of the trials could do this individually (only poss with homogenous data eg mortality)

Question 85

Regression to the mean

Answer 85

People often get better or worse regardless of intervention = not the intervention causing change

Question 86

Equipoise:

Answer 86

Equipoise refers to the situation where the researchers have no preference between the treatments being studied in a trial When it is lost: when those designing the trial or recruiting patients into it do have a preference for one treatment over another, it can be considered unethical to recruit patients into the trial as you are offering some patients what you believe to be a “worse option”