Data Analysis 1: Comparing Means And Medians Flashcards
What is the purpose of performing statistical tests when analysing the data produced in an experiment?
To determine whether sufficient evidence has been generated to enable the experimental hypotheses to be rejected or retained, we need a method of the differences observed between groups/timepoints/conditions are: genuine or meaningful
What is a genuine method?
the difference observed due to a genuine difference between the nature of the varying
experimental conditions, or was it the result of uncontrolled/random variation?
What is a meaningful difference?
the difference constitute an effect-size that is biologically or clinically relevant?
What point does the statistical address?
- provide a method of determining the probability that differences
observed in data sets (e.g. different groups/timepoints) are the result of uncontrolled/random variation, in
contrast to a genuine difference resulting from the varying experimental conditions
What is a p-value?
Q: How are p-values used to interpret results?
A p-value is the calculated probability of observing an effect size as large or larger than the one observed
between the groups, if there was no genuine difference between the groups (i.e. if all of the data points for
the groups being compared actually came from the same group)
How are p-values used to interpret results?
P values are compared to a threshold value (‘α’ value), chosen prior to conducting the study, that is used as
an objective method of determining if an effect size is statistically significant or not.
p=0.05 means….
if there was no genuine difference between the groups, there would be a 5% probability of
observing a difference of at least as large as the one observed.
If p < α, p = 0.01 and α = 0.05, the difference…
…is statistically significant (reject H0)
If p ≥ α, p = 0.10 and α = 0.05, the difference…
…is not statistically significant (accept H0)
The most commonly used alpha value in biomedical science is probably 0.05. Why?
Alpha values used by convention (e.g. 0.05, 0.01) are arbitrary. The problem is that it is ultimately impossible
to know for certain if a result is due to a rare event or a genuine difference, but we still need to make a
decision regarding what the result means.
What should the choice of alpha/critical values consider?
the nature/design of the study and the relative consequences of
Type 1 (false positive) and Type 2 (false negative) errors.
What is a type 1 error?
false positive
What is a type 2 error?
false negative
What is a smaller alpha value more likely to result in?
a false negative
What is a larger alpha value more likely to result in?
a false positive
What is an unpaired test appropriate for analysing?
parametric data from independent samples (unpaired data).
Why is the t-test appropriate for analysing parametric?
the method by which the p-value is calculated assumes the data adheres to a normal distribution
What happens to the factors associated with relatively large p-values?
- difference in between group means decrease
- increase in standard deviation
- decrease in sample size
What happens to the factors associated with relatively small p-values?
- difference in between group means increase
- decrease in standard deviation
- increase in sample size
What is a one-tailed test entail?
Assessing the statistical significance of potential positive and negative differences between one groups and another
What does a two-tailed test entail?
Assessing only the significance of a difference in one specific ‘direction’
What is the main difference between one and two tailed test?
A one-tailed tests will only have one critical region whereas two-tailed tests will have two critical regions
How to know when to use one-tailed or two-tailed test?
A one-tailed test looks for an “increase” or “decrease” in the parameter whereas a two-tailed test looks for a “change” (could be increase or decrease) in the parameter.
What is the result of a two-tailed test differentiate between?
- any change (positive or negative)
- significant difference
What is the result of a one-tailed test differentiate between?
- an increase or a decrease (a change in one direction)
- significant difference
The number of tails evaluated in the analysis should also be reflected in the language used in the hypothesis.
Examples: One-tailed or Two-tailed?
(a) Does treatment with IL-5 have an impact on BAL eosinophil density during an asthma attack?
(b) Is novel drug ‘X’ a more effective treatment for symptoms of ADHD than methylphenidate?
a) two-tailed
b) one-tailed
Parametric methods (e.g. t-test) generate p-values by…
…assuming a specific relationship between the variation of data points to the mean and the probability of observing them
For non-parametric data,how do you generate p values?
A specific relationship doesn’t exist, therefore any test that assumes the relationship exists will generate an inaccurate p-value
Mann-Whitney U test does…
…not assume any particular distribution and it is a non-parametric method.
D
