Finals Review

Question 1

Describe the primary goals of Phase 1 drug trials?

Answer 1

• Goal: Determine which dose of drug is safe and most likely to show benefit
• Estimate largest size of a dose before unacceptable toxicity is experienced by patients (Maximally tolerated dose – MTD)
• Start with a low dose and escalate until a prespecified level of toxicity is achieved

Question 2

Describe what is meant by the efficacy/toxicity trade-off.

Answer 2

A higher dose will likely have a higher efficacy but also a higher toxicity so a Phase I trial works to find the balance of a dose that is high enough to be effective but not have too many toxicity event (the MTD)

Question 3

In a 3+3 design, how is the maximum tolerated dose determined?

Answer 3

Treat 3 participants at dose K
- If no DLT escalate to dose level k+1
- If 2+ DLTs, de-escalate to dose level k-1
- If 1 DLT, treat 3 additional participants at dose level K
– If 1 in 6 DLT, escalate to dose level K+1
 – If 2 in 6 DLT, de-escalate to dose K-1
- MTD is the highest dose where 0 or 1 DLT is observed (repeat as needed)

Question 4

What are some strengths and limitations of the 3+3 design?

Answer 4

Strengths: Easy, small sample, doesn’t require statistician
Limitations:
o Ignores dose history other than previous 3 patients
o Imprecise and inaccurate MTD estimation
o Low probability of selecting true MTD
o High variability in MTD estimates
o Dangerous outcomes

Question 5

What are some strengths and limitations of the Continuous Reassessment Model (CRM)

Answer 5

Strengths: Relatively Precise way to determine the MTD
Limitations: Requires a statistician and modelling, requires assumptions

**ADD TO THIS

Question 6

Write down the null and alternative hypothesis for a futility trial

Answer 6

• Formulation of the null and alternative hypotheses are reversed. Higher alpha (flip alpha and beta so alpha about 0.2 and beta about 0.05)

Continuous:

• Futility Trial. Null: New Tx Mean >= Control Mean + Delta
• Futility Trial. Alt: New Tx Mean < Control Mean + Delta
• Futility Trial. Reject null: New Tx is futile; Do not move forward. Reject is bad.
• Standard. Null: New Tx Mean = Control Mean
• Standard: Alt: New Tx Mean > Control Mean
• Standard. Reject null: New Tx is effective. Reject is good.

Binary:

• Futility Trial. Null: New Tx Prop <= Control Prop - Delta
• Futility Trial. Alt: New Tx Prop > Control Prop - Delta
• Futility Trial. Reject null: New Tx is futile; Do not move forward. Reject is bad.
• Standard. Null: New Tx Prop = Control Prop
• Standard: Alt: New Tx Prop < Control Prop
• Standard. Reject null: New Tx is effective. Reject is good.

Question 7

What are some specific features of futility designs that typically result in a smaller sample size compared to conventional efficacy designs?

Answer 7

• One sided (or historical controls)
• Looking to detect larger differences
• We are less concerned about drawing false-positive ( type I error) conclusions that ineffective treatments may be effective because treatments that are not determined to be futile in phase II would be tested further in phase III trials with smaller error probabilities at the expense of larger sample sizes
• Lower power (higher beta), smaller sample size
• **ADD TO THIS AND MAKE SURE IT’S CORRECT (but basically, proving futility is a smaller claim than proving efficacy so it does not require as many participants)

Question 8

Describe what the differences are in Type I and Type II errors between conventional designs and futility designs.

Answer 8

Conventional: Type I Alpha: Ineffective Therapy is Effective (falsely rejecting H0).

Conventional: Type II error: Effective therapy is ineffective (failed to reject H0).

Futility: Type I: Effective therapy is ineffective (falsely rejected H0 and called an effective treatment futile).

Futility: Type II: Ineffective Therapy is Effective (failed to reject the null).

For futility we have a higher alpha and lower beta

Question 9

In cluster randomized trials, assume there is a positive correlation in the primary outcome for individuals within a cluster. Describe the impact on the alpha level if you ignore the design effect and analyze as if they are independent.

Answer 9

• alpha type I error will increase ; pvalue biased downwards – false positive error rate has increased (alpha)
• POSITIVE INTRACLASS CORRELATION REDUCES VARIATION AMONG MEMBERS OF THE SAME GROUP so failing to acknowledge that will decrease variance which inflates Z/T score, which decreases pvalue

If you ignore the correlation (Which has decreased the variance) you will divide by smaller variance estimate so the t/z statistic will be larger/more extreme so the pvalue will be smaller. So the design effect inflates the variance accordingly to correct the pvalue.

Question 10

Based on a figure of point estimates and confidence intervals, be able to explain whether one would conclude superiority, noninferiority, inconclusive, or inferior treatment.

Answer 10

Blue area is the region of non-inferiority. Usually CI will be pretty large because of small sample size. So unless sample size is very large or margin of non-inferiority delta is too large, it is unlikely to have a scenario “D” where the entire CI is between 0 and delta.

Question 11

27. Be able to define what the margin of non-inferiority is. Also, be able to comment on the choice of this margin compared to an active control’s effect. For example, if an active control has a delta difference with placebo, in a non-inferiority trial for a new treatment compared to that active control, describe how the margin of non-inferiority may compare to this delta and the rationale.

Answer 11

• Margin of non-inferiority Delta: specified in protocol, maximum difference in responses between two interventions that is considered clinical unacceptable (ie* ½ or 1/3 of the established superiority) – retain a certain proportion of the active control’s efficacy
  • Placebo – 40 % mortality
  • AC – 30% mortality
  • Margin – ½ then will accept 5% or 35% mortality
• Non-inferiority is comparing to an active control but we need to consider the effectiveness of active control as compared to the placebo in choosing the margin
•  Must be very valid. Clear protocol and carefully, rigorously conducted (minimal drop out, non-compliance, missing data that might bias the results toward the null in a non-inferiority study because they will look more like “placebo” if they’re not taking a treatment). ESPECIALLY IMPORTANT FOR NON-INFERIORITY BECAUSE IT WILL BIAS TOWARDS THE NULL FOR NON-INFERIORITY WHILE IT WOULD BIAS AWAY FROM THE NULL IN EFFECTIVENESS. This matters because it would lead you to be more likely to call a treatment INCREASES PROBABILITY OF TYPE II ERROR.
• Have to determine the margin of non-inferiority delta – maximum difference between two interventions that is considered clinically acceptable
• • Placebo 40% mortality; Active control 30% mortality; Margin: 35%-30%=5% or 37%-30%=7%? Definitely cannot go to 10% margin because then you’re at placebo. Need to establish what is acceptable.

Question 12

28. Describe why it’s reasonable to plan for testing non-inferiority and then superiority, but not the other way around.

Answer 12

 CANNOT test superiority first and then non-inferiority
• Non-inferiority will have a smaller sample size
o Looking for a smaller effect size for non-inferiority than superiority (superiority is non-inferiority + effectiveness)
 Superiority margin is from intervention vs placebo
• Larger
 Non-Inferiority is for Intervention vs control
• Smaller
o Non-inferiority is one sided while standard/superiority will be two sided.
• Test order has to do with type I error as well. If superiority first, if it fails it would not move on to the non-inferiority test. Has to pass the first test to move to the second.
o If first do non-inf before trying sup. Can detect both non-inf and sup. Non-inf is gate keeper. Then test for sup.
o If do sup first and it fails, we would not move on to non-inferiority. This is due to Type I errors. Can only move on to next sequential test if current test is successful.
• Power issue has to do with the margin and the effect size being smaller (ie* harder to detect) for superiority. Thus sample size must be higher for superiority than non-inferiority. WILL ALSO BE UNDERPOWERED FOR ONE OF THE TESTS IF THE DELTAS ARE NOT EQUAL.

Question 13

Describe the intention-to-treat principle.

Answer 13

As randomized so analyze. If not doing this need to specify and justify

Question 14

Describe the possible effect of dichotomization of a time to event/continuous outcome on the power of a trial.

Answer 14

Loss of power (Would need to increase sample size). Can result in easier to interpret results

Question 15

Describe what is meant by sub-group analyses.

Answer 15

• Who does the treatment work best for
• NIH mandates comparison of sex and racial/ethnic groups
• Usually done to see if the treatment is effective in specific groups (usually for marginally unsuccessful results)
• ALWAYS test for interaction with treatment
  • Don’t interpret the main effects when you’re looking at the interaction
• Still considered post-hoc even when pre-specified
• Only do subgroup analysis if interaction effect is present (usually pvalue will be higher because of lower power for the interaction test so increase alpha to maintain power)

Question 16

What is the role of testing of interactions in sub-group analyses?

Answer 16

who is the treatment most effective for within the population (define from baseline characteristic) ; interaction between baseline variables with the treatment; needs to be prespecified; subgroup needs to be something before treatment (ie* by sex but not by side effect or post treatment mrs or something like that) ; groups cannot be effected by the treatment

Work on this?

Question 17

Describe some ethical reasons for conducting interim analysis?

Answer 17

Detect benefit, safety, harm, or anything strongly indicating one of the treatments might be inferior or ineffective early

Question 18

Describe some administrative reasons for conducting interim analysis?

Answer 18

Be sure the study is being executed as planned, assess the appropriateness of enrollment; find unanticipated problems (ie* non-compliance) that could be correct; check on assumptions (ie* sample size)

Question 19

Describe some economic reasons for conducting interim analysis?

Answer 19

Early stopping for negative results that are wasting resources. Allocation of R&D funds

Question 20

Describe the primary factors that go into the decision to terminate a study early

Answer 20

• Terminate early:
  o Clear early effect – continuing is unnecessary or even unethical
  o Futility – sufficient evidence that the treatments are not different so continuing is unnecessary or unethical
• Impact of early termination on the credibility of the results and the acceptability by the clinical community needs to be taken into account. Mostly for positive results
• Have to weigh the pros and cons of stopping early for either positive or negative
• Issues: Type I error inflation when looking for problems; What to analyze; preliminary information might impact objectivity about the treatment and enrollment (clinicians, patients, etc.) if anything gets disseminated

Question 21

Describe what the differences are between the (1) Haybittle & Peto, (2) Pocock, and (3) O’Brien and Fleming sequential boundaries are with respect to the likelihood of stopping early and for the final look.

Answer 21

????

Question 22

What is the advantage of alpha spending designs over group sequential designs?

Answer 22

-	Alpha – spending function. Fix the total type I error rate 
o	Alpha(t*) = alpha spending function determines how pre-specified alpha is allocated at each interim analysis as a function of the information fraction. Alpha(0)=0 and alpha(1)=1. 
- Group Sequential designs are cumbersome and alpha spending less so
- GSD can have restrictive monitoring times that require equal increments of information (# of patients) and sometimes causes administrative difficulties
- Alpha spending are more flexible than GSD so the idea is to spend/distribute the total prob of false positive risk (type I error) as a continuous function of the information time (alpha spending function)

Question 23

Be able to define and give examples of the various missing data mechanisms: (1) missing completely at random; (2) missing at random; and (3) missing not at random.

Answer 23

• MAR: Probability of measurement not being observed does not depend on what the value would have been. Probability does not depend on the value of Y as Y is missing after controlling for other variables X
• MCAR: Assumption that some data are missing on Y. If the probability of being missing is unrelated to the measurement of Y that would have been observed or other variables X, these data are MCAR
• MNAR: Probability of missing data is dependent on the value of that missing data
  • Non-ignorable – missing data mechanism must be modeled to get reliable parameter estimates. Requires good prior knowledge of cause of missingness – no way to test goodness of fit of this and results may be sensitive to its choice

Question 24

41. Be able to reassign missing values in a table according to best-case and worst-case analyses.

Answer 24

… o Naïve analysis, best-case analysis, worst-case analysis
o If all scenarios pvalues above alpha (Two sample test of equal proportions) – can report as no difference; if all below then there is a difference; issue is if some above and some below (As in example).

Question 25

Be able to explain the Last Observation Carried Forward method for missing data imputation and its main limitation.

Answer 25

• Missing value imputed from a randomly selected similar record
• If longitudinally collected outcomes, LOCF would use the previous outcome for their next missing outcome
• Using a different observations value for imputation. Randomly shuffle and use the first one before that
• Ignores trend over time
• Bad

Question 26

What is the advantage of multiple imputation versus single imputation methods?

Answer 26

• Method for averaging the outcomes across multiple imputed datasets to account for the uncertainty of imputation
  • Missing values are imputated m times to create m data sets with complete data: m is usually 5-10
  • Analysis is conducted on each m datasets leading to m analyses
  • Pooling – consolidate the m results into one result by calculating the mean, variance, and Cis of the param estimates for the variables of concern
• Most popular: Multiple imputation by chained equations (MICE)
• Takes into account the uncertainty of imputation process
• Best

Question 27

What is a SMART trial?

Answer 27

o Adaptive interventions – individualized decision rules
o Look at responders vs non responders and can do further randomization into increased intervention
o Don’t need to power the study for the final comparison of all subgroups
o GOAL: inform the development of the adaptive interventions

Question 28

Explain what the difference between basket, umbrella, and platform trials.

Answer 28

• Basket: Targeted therapy and want to evaluate on multiple diseases that the therapy could be helpful in (should be helpful in all of them). Typically cancer – will help multiple cancers that have the same genetic marker. Multiple diseases should have common genetic marker or characteristic to predict if a patient will respond.
  • Limitation: Assumption that the biomarker is highly predictive
• Umbrella: Evaluate multiple target interventions for a single disease stratified into subgroups by biomarker or other patient characteristics
  • Limitation: Feasibility, especially within a rare disease
• Platform: Allows for interventions to be dropped or added throughout the trial. Multi-Arm, Multi-Stage trials.