Final Exam - Clinical Trials

Question 1

Why does regression to the mean have more impact on the interpretation of uncontrolled (single arm) studies?

Answer 1

Regression to the mean can heavily influence the outcomes of uncontrolled studies because any extreme initial measurements tend to move towards the average upon subsequent measurements. Without a control group, it’s unclear whether changes are due to the intervention or natural variability.

Question 2

Differentiate factors in the design and conduct of a trial that influence internal versus external validity.

Answer 2

Internal validity is influenced by factors within the study that ensure the observed effects are due to the intervention, not confounders (e.g., randomization, blinding). External validity is affected by factors that enable the generalization of the findings to other populations (e.g., participant diversity, setting realism).

Question 3

Clinical trials are designed to answer a specific scientific questions. Briefly describe three aspects of the design that this question depends upon?

Answer 3

Three aspects include: 1) The population targeted, determining who the findings will apply to. 2) The intervention’s specifics, including dosage and duration. 3) The outcomes measured, deciding what effects are being assessed.

Question 4

What defines the study population for a clinical trial?

Answer 4

The study population is defined by specific inclusion and exclusion criteria which may include factors like age, gender, disease stage, and other relevant medical conditions to ensure the population is appropriate for the research question.

Question 5

Briefly describe three important considerations when developing eligibility criteria.

Answer 5

1) Inclusion of a population that can benefit from the intervention. 2) Exclusion of individuals at risk of harm from the study. 3) Selection of participants likely to comply and remain for the study duration with measurable endpoints.

Question 6

Briefly describe one pro and one con that might arise from having restrictive eligibility criteria for a study population that is expected to have the greatest benefit from the intervention?

Answer 6

Pro: Increases the likelihood of observing the intervention’s intended effects. Con: Limits the generalizability of the results to a broader population.

Question 7

Briefly describe one pro and one con that might arise from having broad eligibility criteria for a trial?

Answer 7

Pro: Enhances the generalizability of the study results to a wider population. Con: May dilute the intervention’s observed effects due to heterogeneity in the study population.

Question 8

Briefly describe one pro and one con that might arise from using a run-in period to screen for adherence in a trial of an intervention with anticipated low adherence?

Answer 8

Pro: May improve the quality of the data by ensuring participants are likely to adhere. Con: Could bias the sample by excluding potentially non-adherent participants who might benefit from the intervention.

Question 9

State three conditions under which a run-in period may be beneficial.

Answer 9

1) When assessing adherence is crucial for the intervention’s effectiveness. 2) To assess tolerability of active treatment. 3) To screen for placebo response e.g. in a study on pindolol + fluoxetine versus placebo + fluoxetine for depression, researchers eliminated subjects with 25% improvement in symptoms during single-blind placebo period .

Question 10

Be able to briefly explain the rationale for particular inclusion/exclusion criteria. For example, in a trial evaluating an immunotherapy for lung cancer. Provide some rationale for the following criteria: a. Histologically documented metastatic non-small-cell lung cancer b. ECOG of 0 or 1 with life expectancy 6+ months c. Measurable lesions per RECIST v1.1 criterion d. Willing to stay within 1-hour of treatment site for 28 days

Answer 10

Criteria ensure participants have a confirmed diagnosis (a), are in relatively good health to tolerate the treatment (b), have quantifiable disease progression (c), and can manage the logistical demands of the trial (d).

Question 11

If patients with elevated risk levels are to be randomized in a clinical trial, discuss the factors which would influence the amount of regression toward the mean that would be expected. How can regression toward the mean be reduced?

Answer 11

Factors include the variability and the correlation of measured risk across repeated measures.

To reduce regression, ensure precise measurement methods and consider averaging over multiple measurements.

Including a control group also helps in deducing whether or not the regression is due to treatment or natural variability.

Question 12

A trial is planned to study the effect of drug treatment on CVD morbidity and mortality among participants with systolic BP 130-139 mmHg. One site plans to screen from the general population (e.g. shopping centers) and one site (an HMO) plans to screen all participants who had an elevated BP at their last examination. For which site do you anticipate that regression to the mean will be greater?

Answer 12

Regression to the mean is likely greater at the HMO site, where participants were pre-selected based on elevated BP, which may naturally decrease to average levels upon re-measurement.

Question 13

Be able to explain the difference between the statistical hypotheses underlying superiority versus non-inferiority versus equivalence trials.

Answer 13

Superiority trials test if a new treatment is better than a control. Non-inferiority trials determine if a new treatment is not worse than a control by a pre-defined margin. Equivalence trials assess if a new treatment is statistically similar to a control within a specified range.

Question 14

Be able to propose a non-inferiority margin based on historical effect estimates.

Answer 14

The non-inferiority margin is based on clinical judgement and historical data, aiming to preserve a clinically acceptable portion of the control treatment’s effect. A common approach is that the new treatment must retain 50% of the benefit of active control vs. placebo.

Question 15

Be able to interpret the results of a non-inferiority trial based on a reported 95% confidence interval and the stated margin.

Answer 15

If the 95% confidence interval of the treatment effect difference excludes the non-inferiority margin (does not cross the margin), the new treatment can be considered non-inferior to the control.

Question 16

How does the constancy assumption play a role in our ability to infer superiority of a newly established non-inferior therapy relative to a placebo?

Answer 16

The constancy assumption holds that the effect of the comparator (placebo or active control) remains consistent over time. Violations can lead to incorrect inferences about the new treatment’s superiority.

Question 17

Briefly describe the concept of “biocreep”.

Answer 17

Biocreep refers to the gradual reduction in therapeutic effectiveness of new treatments approved based on non-inferiority to existing treatments, as each generation may only be nearly as good as and possibly worse than its predecessor.

Question 18

A non-inferiority trial of a new treatment (treatment A) is being planned. The active control treatment (treatment B) to be used has been shown to be superior to no treatment – in an earlier trial the point estimate of the relative risk death for treatment B versus no treatment was 0.75. The investigators are planning to power their study to rule out a 25% higher death rate on treatment A versus B. Do you feel this is reasonable?

Answer 18

No, when choosing the non-inferiority margin, a common approach is that the new treatment must retain 50% of the benefit of active control vs. placebo. Thus, wanting to rule out a 12.5% at most is reasonable for a non-inferiority trial margin.

Question 19

Why is an intention-to-treat analysis considered by some to be anti-conservative in a non-inferiority trial?

Answer 19

It includes all participants as originally allocated after randomization, regardless of whether they adhered to the treatment protocol or not. This approach can dilute the treatment effect because it includes data from participants who may not have followed the treatment regimen properly. This can lead to underestimation of the treatment effect relative to the control group. This can potentially lead to a false rejection of the null and conclusion that the new treatment is non-inferior to the standard treatment when it might actually be inferior.

Question 20

1. Define/describe what is meant by intention-to-treat analysis? Describe one pro and one con.

Answer 20

Intention-to-treat analysis is a strategy for analyzing data in which participants are included in the group to which they were originally assigned, regardless of whether they completed the intervention. Pro: Maintains randomization, reducing selection bias. Con: Non-adherence can dilute the estimated treatment effect.

Question 21

2. Define/describe what is meant by per-protocol analysis? Describe one pro and one con

Answer 21

Per-protocol analysis includes only participants who completed the intervention as planned, without significant deviation from the study protocol. Pro: May give a clearer estimate of the treatment effect among adherent participants. Con: Risk of bias from loss of randomization.

Question 22

3. What is an exclusion in the context of analysis? Should an exclusion be included in an ITT analysis?

Answer 22

An exclusion is the removal of participants from the analysis, often due to not meeting the study criteria or for deviations from the protocol.

Exclusions are generally avoided in intention-to-treat analyses to maintain the integrity of the randomization, although post-randomization exclusions can occur for critical issues.

Question 23

4. What is a withdrawal? Should a withdrawal be included in an ITT analysis?

Answer 23

A withdrawal is a participant who decides to discontinue their involvement in the study before it is completed, for any reason. In intention-to-treat analyses, withdrawals are included to preserve the initial random allocation and to minimize the potential for bias.

Question 24

5. Give an example of an “intention to treat”, “modified intention to treat” and “per protocol” or “on treatment: analysis.

Answer 24

In an intention-to-treat analysis, every participant randomized is included in the analysis based on the assigned groups, even if they did not complete the intervention. A modified intention-to-treat analysis might exclude those who never started the intervention but include all others. A per-protocol analysis would only include data from participants who completed the intervention strictly as planned.

Question 25

6. Compare the potential for bias resulting from the exclusion of patients after randomization if they do not meet the eligibility criteria and from the exclusion of patients who did not comply to their assigned treatment.

Answer 25

Excluding patients after randomization because they do not meet eligibility criteria can introduce selection bias, potentially undermining the trial’s internal validity. Conversely, excluding patients who did not comply with their assigned treatment can lead to attrition bias, which may skew the results and provide a misleading estimate of the treatment’s effectiveness.

Question 26

1. Describe what a marginal effect is in the context of an example.

Answer 26

A marginal effect quantifies the expected/ average change in the outcome variable for a one-unit change in a predictor variable, keeping other variables constant.

For instance, in a study of a new drug’s effect on blood pressure, the marginal effect might indicate how much the average blood pressure is expected to change per additional milligram of the drug administered.

Question 27

2. Describe treatment effect heterogeneity

Answer 27

Treatment effect heterogeneity refers to the variation in treatment effectiveness across different groups of participants. This occurs when a treatment does not have a uniform effect on everyone due to individual differences, such as age, genetics, or baseline health status, causing some subgroups to benefit more or less from the treatment than others.

Question 28

3. Briefly describe a typical aim of subgroup analyses

Answer 28

The typical aim of subgroup analyses is to evaluate whether the effect of an intervention is consistent across different predefined subgroups of participants. It seeks to discover if certain characteristics, like demographics or baseline values, influence the effectiveness or safety of the treatment.

Question 29

4. Understand the differences between proper and improper subgroups, and a priori versus post hoc subgroups.

Answer 29

Proper subgroups in clinical trials are based on baseline characteristics, while improper subgroups are based on a post-randomization event.

A priori subgroups are pre-specified, planned before looking at the data, while post hoc subgroups are identified after looking at the data, clearly hypothesis generating.

Question 30

5. Understand the differences between pre- and post-stratification

Answer 30

Pre-stratification refers to the process of grouping participants into subgroups before treatment assignment to ensure balance across treatment arms.

Post-stratification is a statistical adjustment technique used after randomization to correct for imbalances in subgroup distribution or to perform subgroup analyses.

Question 31

6. Be able to interpret a forest plot showing subgroup analyses

Answer 31

A forest plot in the context of subgroup analyses is a graphical representation that displays the estimated treatment effects for each subgroup with their confidence intervals. Interpreting a forest plot involves examining if the confidence intervals overlap with the line of no effect and assessing the consistency of the treatment effect across different subgroups.

Question 32

7. Describe three reasons that result in missing data in clinical trials.

Answer 32

Three common reasons for missing data in clinical trials include participants dropping out, losing contact with participants during the follow-up period, and failure to collect or record some data points due to oversight or technical issues.

In short:
- drop out
- lost contact
- failure to collect/ record

Question 33

8. Be able to distinguish between the three missing data mechanisms (MCAR, MAR, MNAR).

Answer 33

Missing Completely at Random (MCAR), where the likelihood of missing data is unrelated to any study or participant characteristics;

Missing at Random (MAR), where missingness is related to observed data but not the missing data itself; and

Missing Not at Random (MNAR), where the missingness is related to the unobserved data itself.

Question 34

9. Briefly describe two strategies for preventing missing data in clinical trials.

Answer 34

To prevent missing data in clinical trials, two strategies include enhancing participant follow-up procedures to reduce dropouts and designing the study to minimize participant burden, thereby improving adherence and retention rates.

Question 35

10. Be able to describe rationale and strategies for early stopping for harm, futility, and efficacy.

Answer 35

Rationale for early stopping in clinical trials includes stopping for harm if the treatment is found to be unsafe, for futility if interim results suggest that achieving the primary endpoint is unlikely, or for efficacy if significant beneficial effects are observed earlier than expected. Strategies involve predefined stopping rules based on statistical thresholds.

Question 36

11. Understand how O’Brien-Fleming and Pocock group sequential monitoring boundaries differ and their relative pros/cons.

Answer 36

O’Brien-Fleming and Pocock group sequential monitoring boundaries are used in interim analyses of clinical trials. O’Brien-Fleming’s approach requires more substantial evidence to stop a trial early on for efficacy, which is more conservative, while Pocock’s boundaries are uniform throughout the trial, which can lead to more frequent early stopping but with less conservative evidence requirements.

Question 37

12. Describe advantages of an error spending approach for determining group sequential monitoring boundaries.

Answer 37

Provides flexibility in conducting interim analyses at varying points during the trial while controlling the overall type I error rate by ‘spending’ error probability incrementally throughout the trial.

This approach allows trials to adapt to circumstances without compromising statistical integrity.

Question 38

13. Understand the role of a DSMB and importance of their independence and expertise.

Answer 38

A Data Safety Monitoring Board (DSMB) is an independent group of experts that monitors patient safety and treatment efficacy data during a clinical trial. Their independence from the sponsors and researchers is crucial to ensure unbiased oversight and decision-making based on the data.

Question 39

14. Be able to distinguish between conditional and unconditional power, and rationale for monitoring either for futility purposes. Understand what impacts conditional power versus unconditional power.

Answer 39

Conditional power refers to the probability of achieving statistical significance in a trial, given the results observed so far, while unconditional power does not consider the results to date. Monitoring for futility often uses conditional power to decide if a trial should continue. Factors like the effect size and variance observed influence conditional power, while unconditional power is affected by the original assumptions of effect size and variance.

Question 40

15. Why does the test for interaction have lower power than the test for main effects in subgroup analyses?

Answer 40

Tests for interaction have lower power than tests for main effects because they are essentially looking for effects within smaller subgroups, which reduces the sample size available for detecting differences, thus requiring larger effect sizes to achieve the same power.

Also, the complexity involved in an interaction can mean increased variability relative to the main effect, decreasing our ability to detect said interaction effect.

Question 41

16. At a recent interim analysis for a HIV vaccine trial, the DSMB recommended the study be stopped for futility. What does that mean?

Answer 41

If a DSMB recommends stopping a trial for futility, it means the interim analysis suggests that the trial is unlikely to meet its primary endpoints, even if it continues to its planned conclusion.

Question 42

17. What is meant by a treatment x subgroup interaction? Give an example.

Answer 42

A treatment x subgroup interaction occurs when the effect of treatment varies across different subgroups. For example, if a medication lowers blood pressure in younger adults but not in older adults, this difference in effect demonstrates a treatment by age interaction.

Question 43

18. What is the purpose of Data Monitoring Committees (DMCs) (or DMSBs) for clinical trials?

Answer 43

The purpose of Data Monitoring Committees (DMCs) or DSMBs in clinical trials is to provide independent oversight of the trial for patient safety, study integrity, and to evaluate the interim results for early signs of benefit or harm.

Question 44

19. Why is it important for DMCs to be independent? Independent of whom?

Answer 44

It is important for DMCs to be independent of the study sponsors and investigators to avoid conflicts of interest that could bias the assessment of the trial data and to ensure the credibility of the trial outcomes.

Question 45

20. Why is it important to carry out interim analyses in clinical trials?

Answer 45

Interim analyses are important in clinical trials to ensure ongoing participant safety, to evaluate the effectiveness of the intervention early, and to stop the trial if the intervention is proven effective or harmful, or if it is deemed futile to continue.

Question 46

21. Give 2 examples of why trial might be terminated before it is completed.

Answer 46

A trial might be terminated before completion if there is clear evidence of treatment harm or significant benefit, or if it becomes apparent that the trial will not be able to answer the research question due to issues like participant recruitment or retention problems.

In short,
- Safety
- Efficacy
- Futility

Question 47

22. Distinguish between open and closed DSMB reports and describe in a few sentences what the content of each should be.

Answer 47

Open DSMB reports are accessible to the study team. They contain
- Summary data about the trial’s enrollment progress and safety.
- Any data must be presented without grouping by treatment assignment, preserving the masking of all subjects.
- Outcome results are not generally included.

Closed reports are confidential and accessible only to the unblinded statistician and DSMB members.
- Grouped safety data and, if appropriate, efficacy data are available.
- Grouped data should be presented by coded treatment arm.

Question 48

23. Why are statistical stopping guidelines used for trials?

Answer 48

Statistical stopping guidelines are used to ensure that decisions to stop a trial early are based on pre-established rules that maintain the integrity of the study’s results while also respecting resource limitations and ethics.

Question 49

24. Give an example of a stopping guideline.

Answer 49

An example of a stopping guideline could be a predefined boundary in an interim analysis, such as the O’Brien-Fleming boundary, which would call for trial cessation if a clear benefit or harm is observed that crosses this statistical threshold.

Question 50

25. The Lan-Demets spending function approach requires specification of the “information fraction” at each interim analysis. What does this mean?

Answer 50

The Lan-Demets spending function approach involves calculating the proportion of the total information (e.g., the number of events, participants enrolled) that has been accumulated at each interim analysis. This helps to control the overall type I error rate while allowing for flexibility in the timing and number of interim analyses.

Question 51

1. Understand how variability, effect size, type I and II error rates impact sample size requirements.

Answer 51

Variability, effect size, and type I and II error rates are crucial in determining the required sample size for a study. Greater variability or a smaller effect size requires a larger sample size to detect a true effect. Lower type I (probability of a false positive) and type II (probability of a false negative) error rates also necessitate a larger sample size to achieve sufficient power to detect an effect if one truly exists.

Question 52

2. Define type I and II error rates. Define power.

Answer 52

Type I error rate is the probability of incorrectly rejecting the null hypothesis when it is true (false positive), commonly set at 0.05. Type II error rate is the probability of failing to reject the null hypothesis when it is false (false negative), with the complement being the power of the study, which is the probability of correctly detecting an effect if there is one.

Question 53

1. Describe two important principles when specifying an effect size?
2. In a parallel group study in which a continuous response variable is to be used (e.g., systolic blood pressure (SBP) change after 24 months), for fixed level of the hypothesized treatment effect (e.g., a difference of 5 mmHg), how does sample size vary according to the standard deviation for SBP change.

Answer 53

1. When specifying an effect size, it is important to consider the clinical relevance of the effect size and the precision of the measurements.
2. If the standard deviation for SBP change is large, a larger sample size will be needed to detect the fixed effect of 5 mmHg as significant, since the natural variability could mask the treatment effect.

Question 54

4. How might poor adherence affect the hypothesized difference between the treatment and control groups in a parallel groups randomized trial? How might this be accounted for in a sample size calculation?

Answer 54

Poor adherence can reduce the observed difference between the treatment and control groups in a parallel groups randomized trial because it may dilute the treatment effect. This can be accounted for in sample size calculations by increasing the estimated sample size to compensate for the anticipated lack of adherence.

Question 55

5. Many investigators argue that in scientific papers more use should be made of confidence intervals (as opposed to P-values) and methods sections should include a discussion of Type I and Type II error rates. Why is this particularly important in “negative” trials (trials in which a significant treatment difference is not observed)?

Answer 55

Confidence intervals provide a range of values within which we can be confident the true effect lies and are informative regardless of the p-value.

In negative trials, discussing Type I and Type II error rates is important because it clarifies the study’s ability to detect a difference, if one exists, and the likelihood of the null findings being due to chance.

Question 56

6. In a number of primary prevention trials for CVD the event rate in the control group was much less than originally planned. What impact on the experiment would you expect this to have?

Answer 56

If the event rate in the control group of CVD prevention trials is lower than expected, the actual difference between treatment and control groups is smaller than the expected difference. This means a larger sample size than planned is required to achieve the desired power, meaning the trial is likely underpowered.

This will likely impact cost since study duration might be extended to collect a sufficient number of events in a larger sample size than originally planned for.

It also raises questions in terms of generalizability.

Question 57

7. In many trials the rate of non-compliance is greater early in follow-up as compared to later. If the expected event rate in the experimental treatment group is expected to increase over follow-up, what impact will this have on sample size as compared to a constant rate of non-compliance, assuming the cumulative non- compliance over all of follow-up is 30%?

Answer 57

If the rate of non-compliance is higher early on but the expected event rate in the treatment group increases over time, the initial non-compliance might have less impact on the final results.

However, accounting for variable non-compliance and event rates over time can complicate sample size calculations and may require a larger sample size early on (and smaller sample size later on) to maintain statistical power at a particular level over time.

Question 58

8. Discuss why it is important in a clinical trial with a planned intention to treat analysis to consider compliance to treatment in the design?

Answer 58

Considering compliance to treatment in the design of a clinical trial is important, especially for intention-to-treat analysis, because non-compliance can lead to underestimation of the treatment effect. Design features such as over-enrollment, strategies to improve adherence, and appropriate statistical methods can mitigate the impact of non-compliance.

Question 59

1. Understand the concept of equipoise and its relation to the ethics of randomization

Answer 59

Equipoise refers to the genuine uncertainty within the scientific and medical community regarding the comparative therapeutic merits of each arm in a trial. It’s a fundamental ethical prerequisite for randomization because it means that no participant is knowingly given a substandard treatment. Without equipoise, randomizing patients could be ethically questionable, as it may imply giving some patients a treatment known to be inferior.

Question 60

2. Provide three key elements to ensure informed consent.

Answer 60

To ensure informed consent, three key elements are necessary:
- comprehension, where the participant must understand the study’s nature, purpose, and potential risks;
- voluntariness, free of coercion or undue influence; and
- full disclosure, meaning the participants are provided with all information relevant to their decision to participate, including risks, benefits, and alternatives.

Question 61

3. Provide three reasons it may be considered unethical to continue an ongoing randomized trial.

Answer 61

Continuing an ongoing randomized trial may be considered unethical
- if new evidence emerges indicating that the current standard of care has been surpassed or is harmful,
- if the trial objectives have already been met or cannot be achieved, or
- if the risk-benefit ratio has shifted unfavorably, potentially exposing participants to harm or ineffectiveness.

Question 62

4. Briefly explain the concept of distributive justice.

Answer 62

Distributive justice in the context of clinical trials refers to the fair distribution of the risks and benefits of research. It demands that no segment of the population should bear an undue burden of research or be unjustly excluded from its benefits, ensuring equitable access to participation in research and its potential benefits.

Question 63

5. Provide three examples of conflicts of interest that a clinical trial investigator may experience.

Answer 63

Conflicts of interest for a clinical trial investigator could include
- financial incentives from a pharmaceutical company sponsoring the trial,
- the desire for positive results to support a prior hypothesis or academic advancement, and
- personal beliefs that may bias the conduct and outcomes of the trial.

Question 64

6. Describe a reason lack of representation in clinical trials can be problematic.

Answer 64

Potential reasons: Mistrust in science, or geographic or economic inability to participate in the trial.

Lack of representation in clinical trials can be problematic because it may lead to findings that are not generalizable to the entire population, especially underrepresented groups.

This can result in treatments that are
- less effective or
- have unanticipated side effects in populations that were not adequately included in the trial phases.