Multiplicity of Data: Multiple Outcomes/Treatments and Repeated Measures

Question 1

How can the issue of multiplicity be addressed with regards to multiple outcomes?

Answer 1

A priori triage of outcomes (primary and secondary)
Composite outcomes
Adjust for multiple comparisons

Question 2

How can the issue of multiplicity be addressed with regards to multiple arms?

Answer 2

Global hypothesis
Key comparisons only
Adjust for multiple comparisons

Question 3

How can the issue of multiplicity be addressed with regards to multiple subgroup comparisons?

Answer 3

Fewer pre specified sub-groups with rationale

Interaction tests

Adjust for multiple comparisons

Question 4

How can the issue of multiplicity be addressed with regards to repeated time points of measuring outcomes?

Answer 4

Pre-specified key time point
Special statistical methods
Summary measures used
Adjust for multiple comparisons

Question 5

How can the issue of multiplicity be addressed with regards to interim analyses?

Answer 5

Statistical stopping rules

Question 6

Describe two problems caused by multiplicity of data

Answer 6

Increased false positive rate and difficulties reporting and interpreting results of trials

Question 7

List four approaches that can be used to deal with multiplicity issues when you have multiple outcomes

Answer 7

Specify the priorities of the endpoints in advance, choose a combined endpoint, adjust the threshold for determining statistical significance, or combine into a single summary statistic.

Question 8

Describe the Bonferroni method for adjusting threshold for statistical significance in the case of multiple tests performed

Answer 8

With the Bonferroni correction, the significance level for each individual test is set to be the overall significance level (usually 5%) divided by the number of tests performed. At least one of the tests needs to produce a p-value below this level to claim an overall significant treatment difference i.e. to reject the global null hypothesis that there is no difference in the treatment effect across all the outcomes tested.

Question 9

Describe the advantages and disadvantages of the Bonferroni method

Answer 9

The Bonferroni method benefits from being easy to implement and interpret. However, there are a number of problems with this approach. It implies all outcomes have equal priority, it has low power, the adjustment is usually too conservative (i.e. harder to declare statistical significance) and it assumes that the outcomes are not correlated