ICH E9 Clinical Trial Statistics Flashcards
Bias, Definition in Clinical Trials, Sources (4)
The systematic tendency of any factors associated with the:
- design (e.g., lower risk subject receives more placebo)
- conduct (e.g., protocol violation and subject exclusion)
- analysis and
- interpretation
of the results of clinical trials to make the estimate of a treatment effect deviate from its true value.
Robustness, Definition in Clinical Trials
The sensitivity (tolerability) of the overall conclusion to various limitations of the data, assumptions, and analytic approaches to data analysis.
High robustness implies treatment effect and primary conclusion of the trial are not substantially affected by the alternative assumption or analytic approaches.
Development Plan, Broad Aim
Determine whether there is a dose range and schedule at which the drug can be shown to be simultaneously safe and effective, to the extent that the risk-benefit relationship is acceptable ( for the subject and indication).
Clinical Development, Considerations
Confirmatory vs Exploratory Trials
Confirmatory trials: Hypothesis testing of predefined hypothesis. Significant results are sufficient to demonstrate efficacy.
Exploratory trials: Hypothesis generating from data and testing of post hoc hypothesis. Significant results contribute but cannot be the sole basis of the formal proof of efficacy.
Clinical Development, Scope of Trials (7)
- Population: from more defined to more relaxed inclusion/exclusion criteria to approach the indicated population.
- Primary and Secondary Variable: predefined, clinically relevant, scientifically valid main and supportive results
- Composite Variables: Integrated various measurements into a single variable (index) using a predefined algorithm (e.g., rating scales for arthritis evaluation)
- Global Assessment Variables: Recapitulate clinician assessment of the overall safety and efficacy, reported a scale of ordered categorical ratings (poor, normal, well).
- Multiple Primary Variables: A predefined sets o results, each or a subset of which are sufficient to cover the range of therapeutic effects.
- Surrogate Variables: Indirect criteria with predictive power used in lieu of clinical assessment due to practicality.
- Categorized Variables: Predefined and specified dichotomization/categorization of continuous or ordinal variables
Clinical Development, Designs Techniques to Avoid Bias (2)
- Blinding: intended to limit the occurrence of conscious and unconscious bias in conduct and interpretation of trial.
- Randomization: introduces a deliberate element of chance into the treatment assignment to avoid bias arising from predictability in clinical trials.
Clinical Development, Design Configuration
- Parallel Group Design
- Crossover Design
- Factorial Designs
Multicenter Trials, Statistical Analysis for Multicenter, Treatment Effects
Heterogeneity vs Homogeneity:
Treatment x Center Interaction Test
Heterogeneity and homogeneity across centers should be considered in statistical analysis:
Heterogeneous treatment effect across centers:
Treatment x Center Interaction is needed, else the main effect is controversial. (Heterogeneity should be explained by analysis or trial.)
Homogeneous treatment effect across centers:
Treatment x Center Interaction reduces efficiency of the test for main effect.
Note: Treatment-by-Center Interaction isolates effect of each center instead of treating all centers as a whole.
Multicenter Trials, Statistical Analysis for Multicenter, Treatment Effects
Fixed Model vs Mixed Models
Fixed Model: Assume center and treatment-by-center effect to be universal. Heterogeneity of centers are observed and explained.
Mixed Models: Consider center and treatment-by-center effect to be random. Heterogeneity of centers are integrated in the analysis.
Note: Mixed model are especially relevant when the number of sites is large.
Trial Statistics, Type of Comparison
Statistics of trials to demonstrate:
- superiority (i.e., relative to placebo, active comparator, or various dose)
- equivalence (e.g., bioequivalence trials, two sided) and non-inferiority (e.g., active control trials, one sided)
- dose-response relationship (all trial stages)
Statistical Analysis, Equivalence and Non-inferiority Trials, Statistical Test Precautions for Type I Errors
Inappropriate method
Appropriate method
To avoid type I errors, or falsely claiming equivalence or non-inferiority, the statistical analysis:
MUST NOT base observing a non-significant test results of the hull hypothesis that there is no significant difference between the investigational product and the activator.
Instead, it SHOULD based on one-sided test to demonstrate the distribution of investigational product effect is significantly above the lower equivalence margin for non-inferiority (and simultaneously significantly below the higher equivalence margin for equivalence)
Statistical Analysis, Group Sequential Designs
Grouping subject outcomes for assessment at periodic intervals, commonly used in interim analysis of large long-term trials (i.e., Kaplan Meier analysis for mortality)
Sample Size, Power Analysis
General Considerations: alpha and beta levels
Special Considerations: Group Sequential Design
Conventional standards:
Type I error: < 0.05
Type II error: < 0.1 or < 0.2
In group sequential design:
re-estimation of sample size is possible, when effect size and variability deviate from the original estimates.
Clinically Acceptable vs Clinically Relevant
Clinical Acceptable: Describe the significant agreement between the groups that makes clinically indistinguishable in equivalence and non-inferiority trials.
Clinically Relevant: Describe the significant difference between the groups which are clinically meaningful in superiority trials
Data Capture and Processing, Important Considerations NA
Must ensure distinguishing:
Missing Values,
Value “0”, and
Characteristic Absent
Trial Conduct Considerations, Trail Monitoring and Interim Analysis, Monitoring Types (2)
Types of monitoring charactering confirmatory clinical trials by companies
- Oversight of the quality of the trial: no unblinding required and no risk of bias.
- Analysis for treatment comparison (i.e., interim analysis): requires unblinding, necessitates statistical plan in protocol.
Trial Conduct Considerations, Changes in Inclusion and Exclusion Criteria, Scenarios (3)
Inclusion and exclusion criteria generally remains as pre-specified, except for where justified:
- growing internal/external medical knowledge
- regular violations of entry criteria by staff
- low recruitment rates due to over-restrictive criteria
Inclusion and exclusion criteria necessitates protocol amendment and approval.
Trial Conduct Considerations, Accrual Rates
Appreciably falling accrual rate must be remediated to protect the power of the trial.
Trial Conduct Considerations, Sample Size Adjustment
The potential need for re-estimation of the sample size should be envisaged in the protocol.
Trial Conduct Considerations, Interim Analysis and Early Stopping, Early Termination Scenarios by Interim Analysis with Group Sequential Design (3)
The goal of such an interim analysis is to stop the trial early if:
- Superiority of the treatment is clearly established (requires more evidence)
- demonstration of a relevant treatment difference has become unlikely
- unacceptable adverse effects are apparent (requires less evidence)
Trial Conduct Considerations, Interim Analysis and Early Stopping,
Fixed Alpha vs Flexible Alpha Spending Function Approach
To control for Type I error,
Fixed Alpha: Predefined Alpha level divided equally among interim and final analysis based on preliminary data (can be too inefficient or too conservative if true effect size is greater or smaller than expected).
Flexible Alpha: Varying alpha over time and across interim and final analyses based on accumulating data.
Trial Conduct Considerations, Interim Analysis and Early Stopping,
Unplanned Interim Analysis Disclosure (4)
Unplanned Interim Analysis should be avoided, as it may flaw the results of a trial and possibility weaken confidence in the conclusion drawn. However, if conducted the clinical study report must disclose:
- Reason for why it was necessary
- Extended of unblinding
- Magnitude of the bias introduced
- Impact on the interpretation of result
Trial Conduct Considerations, Role of Independent Data Monitoring Committee (IDMC)
Recommends whether to continue, modify or terminate a trial based on interim analysis of trial progress, safety and efficacy data.
Note: Independent nature of IDMC is intended to control the sharing of important comparative information and to protect the integrity of the clinical trial.
Data Analysis Considerations, Pre-specification of the Analysis
Confirmatory vs Exploratory Trials
The statistical section of the protocol should include principals of the statistic analysis, for
Confirmatory trials: all the analysis pre-specified in protocol (and amendments if needed)
Exploratory trials: more general principles and directions
Data Analysis Considerations, Analysis Sets
Principles for the Analysis Set Selection (2)
Unless data on all subjects are complete (which is rarely the case), decisions concerning the analysis set should be guided by the following principles to select analysis set:
- to minimize bias
- to avoid inflation of type I error
Data Analysis Considerations, Analysis Sets
Full Analysis Set, Practical Definition
In practice, full analysis set, describes the analysis set which is as complete as possible and as close as possible to the intention-to-treat ideal of including all randomized subjects.
** Intention-to-treat principal: primary analysis should include all randomized subjects.
Data Analysis Considerations, Analysis Sets
Full Analysis Set, Justified Exclusion, Scenarios (3)
- eligibility violations (failure to satisfy major entry criteria)
- compliance violations (failure to take at least one dose of trial medication)
- Other variable violations (lack of any data post randomization)
Data Analysis Considerations, Analysis Sets
Full Analysis Set,
Justified Exclusion by Eligibility Violations, Justified Scenarios (4)
Only under following circumstances, the subject may be excluded without the possibility of introducing bias:
- the entry criterion was measured prior to randomization
- the detection of the relevant eligibility violations can be made completely objectively
- all subject receive equal scrutiny for eligibility violation
- all detected violation of the particular entry criterion are excluded
Data Analysis Considerations, Analysis Sets
Full Analysis Set,
Partial Medication Compliance, Scenarios (3)
- No medication taken: Ok to exclude if non-compliance is not due to specific assignment
- Withdrawn after some medication: include as per the intention-to-treat principle (analysis with imputation, modeling, and qualitative techniques requires justification).
- Complete medication regimen: Include as per the intention-to-treat principle
Data Analysis Considerations, Analysis Sets
Per Protocol Set, Criteria (3)
“Per protocol set” = “valid case” = “efficacy sample” = “evaluable subjects”:
a subset of the subjects in the full analysis set who are more compliant with the protocol specified criteria:
- the completion of a certain pre-specified minimal exposure to the treatment regimen
- the availability of measurements of the primary variables
- the absence of any major protocol violations including the violation of entry criteria.
Data Analysis Considerations, Analysis Sets
Roles of the Different Analysis Sets
Selection of Full vs Per Protocol Set
For Superiority vs equivalence/non-inferiority Trials
Comparing results from Full vs Per Protocol sets explores the sensitivity of conclusions to the choice of the set of subjects analyzed.
Superiority Trial: Full set analysis
Equivalence/non-inferiority Trail: Per protocol analysis
Data Analysis Considerations, Missing Values and Outliers
Missing values and outliers are typically inevitable in practice. Therefore, the methods for handing missing values and outliers need to be pre-defined in protocol and refined during the blind review.
Data Analysis Considerations, Data Transformation
Data transformations (square root, log) and data derivatives (delta, %change, AUC, ratio) should conform to the standards in the specific clinical areas.