HSPH PBL 1: Critical Appraisal

Question 1

What is meant be ‘randomised’ in an experiment?

Answer 1

Where all participants have the same chance of being in either the treatment or control groups; this means that it is likely that the two groups should be similar in all respects apart from the treatment they receive

Question 2

What is a primary outcome?

Answer 2

Most important outcome of the trial which will influence the size of the trial required and whether the trial needs to be stopped early

Question 3

What is an intention to treat analysis?

Answer 3

In a trial, a participant may not receive the treatment to which they are allocated; but in this form of analysis the results are still analysed according to the treatment that participant was initially assigned to

Question 4

What is an on-treatment analysis?

Answer 4

Where the results are analysed according to the treatment the participant actually received, not the random group they were assigned to, and therefore the randomisation of the trial is not preserved and doesn’t necessarily reflect what happens in practice most effectively.

Question 5

What is the data and safety monitoring committee?

Answer 5

Independent committee that meets to review the trial progress and to ensure the trial is stopped early if there is evidence of treatment benefitting or harming the patients

Question 6

Define ‘hazard ratio’

Answer 6

Compares the risk of an outcome occurring over a specific time period in the two trial arms

Question 7

Define ‘relative risk’

Answer 7

Number of outcomes that have occurred and are compared between the two trial arms (no length of time for events to occur is taken into consideration)

Question 8

What is a ‘current controlled trials number’?

Answer 8

Funding bodies and journals require all clinical trials to be registered prospectively (in advance) in order to ensure that society is aware of the current investigations planned so opportunities for collaboration and duplication of research are facilitated

Question 9

What is a STEMI?

Answer 9

ST elevation myocardial infarction is a heart attack that results in the ST segment of the ECG trace being higher than the baseline

Question 10

What is a PCI?

Answer 10

A non-surgical procedure used to treat the stenotic coronary arteries of the heart by insertion of a deflated balloon via catheter up through blood vessels until they reach the site of the blockage. At the site of the blockage the balloon is inflated in order to open the artery and facilitate blood flow, a stent is often placed at this site also to permanently open the artery to prevent further ischaemic attacks.

Question 11

What is refractory angina?

Answer 11

Chest pain due to ischaemia of the heart muscle that is resistant to therapy

Question 12

What is standard error?

Answer 12

The standard deviation of the sampling distribution.

Question 13

How can you calculate standard error?

Answer 13

Standard error = standard deviation/ square root of n

Question 14

How can you calculate a 95% confidence interval of a mean?

Answer 14

Sample mean +/- 1.96 x Standard Error of the Mean

Question 15

How can you calculate a 95% confidence interval of a difference of means?

Answer 15

The difference in means +/- 1.96 x standard error of the difference in means

Question 16

How can you calculate a 95% confidence interval of relative risk?

Answer 16

Log (Relative Risk) +/- 1.96 Standard Error of the log(Relative Risk)

Question 17

What is a p-value?

Answer 17

The probability that we would obtain this result by chance if in reality the intervention was entirely ineffective

Question 18

What is meant by ‘statistical significance’?

Answer 18

This is usually accepted to be at a 5% level, and therefore a significant result is one where the p value calculated for the results is less than 0.05, and one that is not significant is where p > 0.05

Question 19

How would you interpret the data if a confidence interval for a relative risk contains 1?

Answer 19

That the p value is >0.05, and is not statistically significant as there is a chance that there is no difference

Question 20

How would you interpret the data if a confidence interval for a relative risk doesn’t contain 1?

Answer 20

That the p value <0.05, and is statistically significant

Question 21

How would you interpret the data if a confidence interval for a difference of means contains 0?

Answer 21

That the p value >0.05 and is not statistically significant

Question 22

How would you interpret the data if a confidence interval for a difference of means doesn’t contain 0?

Answer 22

That the p-value <0.05 and is statistically significant

Question 23

Why are critical appraisals of trials conducted?

Answer 23

To check whether a sensible question has been asked and answered, to determine bias, to determine relevance and importance of results and determine whether the results are applicable to local patients

Question 24

What is the research question for this trial?

Answer 24

To determine whether performing preventive PCI as part of the procedure to treat the infarct artery would reduce the combined incidence of death from cardiac causes, nonfatal myocardial infarction or refractory angina

Question 25

What is the primary outcome for this trial?

Answer 25

Death from cardiac causes, non-fatal myocardial infarction or refractory angina

Question 26

What is the secondary outcome for this trial?

Answer 26

Death from non-cardiac causes and repeat revascularisation procedures such as PCI or CABG

Question 27

How was the randomisation conduction?

Answer 27

Computer-generated in blocks of four at each study centre

Question 28

Was the sample size appropriate?

Answer 28

Yes, as there was a calculation that a sample of 600 would provide a sample power of at least 80% to detect a reduction in risk of 30% of the patients at a 5% level of significance, however a sample this large wasn’t required after a recommendation from the data and safety monitoring committee based on the highly significant difference between the treatment and non-treatment groups with the pre-existing sample of 465

Question 29

How were the patients lost to follow up dealt with?

Answer 29

• 10 patients in the preventive-PCI group were lost to follow-up for non-fatal events
• 8 in the non-preventive PCI group were lost to follow up for non-fatal events
All patients in the study were registered with the Medical Research Information Service so that should the patients die, they would receive the death certificate to document this evidence

Question 30

How may non-compliance have affected this study?

Answer 30

Unlikely to affect the results as all deaths will still be recorded

Question 31

How was blinding conducted in this study?

Answer 31

Single blind study – the patients did not know what they received but the doctors/surgeons did.
However, there were independent cardiologists and cardiac surgeons used who were not notified about the study group assignments and these were involve in assessing the specified primary and secondary outcomes

Question 32

Interpret the confidence interval for this study.

Answer 32

The primary results are that the hazard ratio in the preventive-PCI group was 0.35 with a confidence interval of 0.21-0.58 with a P<0.001. This means that patients who received the preventive-PCI had a 65% reduced risk of the primary event than patients who did not receive preventive PCI with a confidence interval of 42-79% reduction, and therefore we are 95% confident that the true effect of preventive PCI in the whole population of patients lies between these values of reduction.

Question 33

Interpret the p-value for this study

Answer 33

The probability that the hazard ratio difference between the preventive and non-preventive PCI occurred entirely due to chance is less than 1 in 1,000 (p < 0.001) and therefore it would be extremely unlikely that this is the case, and therefore that the treatment does have an effect.

Question 34

Explain the size of the treatment effect from this study

Answer 34

The primary outcome occurred in 21 patients in the preventive-PCI group and 53 in the group receiving non-preventive PCI. This corresponds to 9 events per 100 in the preventive PCI and 23 per 100 in non-preventive PCI; this illustrates an absolute risk reduction of 14 percentage points which is significant

Question 35

Explain the clinical significance of the results

Answer 35

There is an upper limit of a 95% confidence interval that implies you are fairly certain to reduce the risk of the primary outcome by at least 42% (up to 79%) which is clinically important.

Question 36

Should the preventive PCI method be adopted as a result of this study?

Answer 36

There is clear evidence that preventive PCI does reduce the risk of subsequent adverse cardiovascular events compared with PCI limited to the infarct artery. Two other studies had consistent results, but these were not significant due to the studies being too small, and therefore lacking power (A26). Therefore, this intervention should be adopted.

Question 37

What are the drawbacks of an on-treatment analysis?

Answer 37

Therefore the randomisation of the trial is not preserved and doesn’t necessarily reflect what happens in practice most effectively. This form of analysis can be seriously biased in either direction if more than a tiny percentage change from the randomisation group

Question 38

What is a confidence interval?

Answer 38

The limits of uncertainty that a sample mean reflects the true mean of the whole population. 95% confidence interval means that there is a 95% chance that the confidence interval contains the true population mean.