Statistical hypothesis testing Flashcards
What is the purpose of using comparison of groups in research?
- can establish effectiveness if new Rx
- make Dx
- risk factors for disease
- characteristics for disease
What is null hypothesis significance testing (NHST?
= statistical hypothesis testing
comparison of null and alternative hypothesis generated P-value
What is the function of a P-value?
used to ascertain whether sample data provides evidence for difference between the groups in the population
What are the key features of an RCT study design?
Samples taken from a given population (ideal: that this sample is representative)
Parallel study design
Sample groups are RANDOMLY allocated to either intervention/exposure group or to placebo/non-intervention group
Compare treatment efficacy at the end time point between 2 groups
May also do another leg of RCT study where the groups/interventions are swapped
How is the ‘sample mean difference’ in a RCT calculated?
Sample mean difference is an ESTIMATE for POPULATION PARAMETER
calculated either by:
a) intervention - control
or
b) control - intervention
numerical value will be the same for both, just if it is ±
What do statistical hypothesis testing measure?
the extent to which the study sample estimate (i.e. the data) reflects the difference for the relevant (wider) population
measured by the P-value
What questions are we trying to answer by using statistical hypothesis testing?
Does the ‘sample estimate’ support the assumption?
Does ‘sample estimate’ reflect a difference? (Data is used as evidence here)
What is meant by ‘equipose’ as the starting point for statistical hypothesis testing?
equipose = null hypothesis
there is no difference between the 2 sample groups
this is the traditional starting point for statistical hypothesis testing
question we’re trying to answer through stats is does the data/sample estimate go in favour of null hypothesis (H0) or is in in favour of an alternative hypothesis (where there is a difference)
What are the main formal steps in performing statistical hypothesis testing?
LEARN **
- Define statistical null hypothesis and alternative hypothesis
- obtain (representative) sample from the population
- Calculate value of the test statistic (using sample) specific to null hypothesis (using data)
- Use test statistic to derive probability (P-value) that quantified whether null hypothesis should be supported or rejected
- Interpret probability (P-value) and sample data
When are the null and alternative hypotheses defined relative to the study?
Should be be defined prior to collecting sample data
How do statistical and research hypotheses differ?
Statistical hypotheses are very formal
Research hypotheses may result from a pre-conceived idea of a direction or association
(usually informed by prior data). These are usually postulated and inform in experimental design
What is a null hypothesis?
= H(0)
in population where sample taken, there is no difference between the intervention and control groups
in terms of mean data values
What does statistical hypothesis testing usually include?
- extent to which sample estimates supports null hypothesis
- measured by probability (= P-value)
- evidence difference in mean effectiveness (intervention vs. cntrl) in population
What is the alternative hypothesis?
= H(A) In population where sample taken, there is a difference between intervention and control groups such that either Int > Control OR Int < Ctrl
Why is the alternative hypothesis considered to be a 2-sided concept?
there are 2 potential outcomes:
Int > Ctrl
OR
Ctrl > Int
How does P-value inform the decision on whether sample estimate lends support to Null or alternative hypotheses?
P-value is a probability
between 0 < P-value < 1.0
Strength of supporting null hypothesis:
- LIKELY: P-value = 1.0
- UNLIKELY: P-value = 0.0
What is the P-value?
probability of OBTAINING sample difference in mean data point under null hypothesis
i.e. difference in mean data points between groups = 0.00
What is the relationship between the P-value and supporting the null hypothesis?
large P-value (near 1.0) MORE LIKELY to support null hypothesis
small P-value ((near 0.0) LESS LIKELY to support null hypothesis
(^ more likely to support alternative hypothesis)
Why is the P-value conceptually difficult in most study designs?
we usually only have one event, but P-value is in its truest form based on repeated sampling
What is the (widely used)
‘critical level of significance?’
P=0.05 cut-off
i.e. 5% significance
What is the practical caveat of using P=0.05 as the critical level of significance?
It is treated as a dichotomy
e.g. P<0.05 null hypothesis is automatically rejected and the difference in sample data is taken to be as important clinically as well as statistically
this is not always the case
What is the main issue with using statistical hypothesis testing?
People often take statistical significance as a proxy for clinical significance
(rarely one can substitute another. P-value should not be used to assess for clinical significance)
What inference can be made using the P-value?
Statistical significance ONLY
not clinical significance
What is the P-value based on?
- theoretical concept
- using repeated sample from the population
- using the same sample size and conditions for each repeat
- each sample should have a unique mean difference
- there should be variation between sample estimates
