Data Analysis 1: Comparing Means And Medians

Question 1

What is the purpose of performing statistical tests when analysing the data produced in an experiment?

Answer 1

To determine whether sufficient evidence has been generated to enable the experimental hypotheses to be rejected or retained, we need a method of the differences observed between groups/timepoints/conditions are: genuine or meaningful

Question 2

What is a genuine method?

Answer 2

the difference observed due to a genuine difference between the nature of the varying
experimental conditions, or was it the result of uncontrolled/random variation?

Question 3

What is a meaningful difference?

Answer 3

the difference constitute an effect-size that is biologically or clinically relevant?

Question 4

What point does the statistical address?

Answer 4

• provide a method of determining the probability that differences
  observed in data sets (e.g. different groups/timepoints) are the result of uncontrolled/random variation, in
  contrast to a genuine difference resulting from the varying experimental conditions

Question 5

What is a p-value?

Answer 5

Q: How are p-values used to interpret results?
A p-value is the calculated probability of observing an effect size as large or larger than the one observed
between the groups, if there was no genuine difference between the groups (i.e. if all of the data points for
the groups being compared actually came from the same group)

Question 6

How are p-values used to interpret results?

Answer 6

P values are compared to a threshold value (‘α’ value), chosen prior to conducting the study, that is used as
an objective method of determining if an effect size is statistically significant or not.

Question 7

p=0.05 means….

Answer 7

if there was no genuine difference between the groups, there would be a 5% probability of
observing a difference of at least as large as the one observed.

Question 8

If p < α, p = 0.01 and α = 0.05, the difference…

Answer 8

…is statistically significant (reject H0)

Question 9

If p ≥ α, p = 0.10 and α = 0.05, the difference…

Answer 9

…is not statistically significant (accept H0)

Question 10

The most commonly used alpha value in biomedical science is probably 0.05. Why?

Answer 10

Alpha values used by convention (e.g. 0.05, 0.01) are arbitrary. The problem is that it is ultimately impossible
to know for certain if a result is due to a rare event or a genuine difference, but we still need to make a
decision regarding what the result means.

Question 11

What should the choice of alpha/critical values consider?

Answer 11

the nature/design of the study and the relative consequences of
Type 1 (false positive) and Type 2 (false negative) errors.

Question 12

What is a type 1 error?

Answer 12

false positive

Question 13

What is a type 2 error?

Answer 13

false negative

Question 14

What is a smaller alpha value more likely to result in?

Answer 14

a false negative

Question 15

What is a larger alpha value more likely to result in?

Answer 15

a false positive

Question 16

What is an unpaired test appropriate for analysing?

Answer 16

parametric data from independent samples (unpaired data).

Question 17

Why is the t-test appropriate for analysing parametric?

Answer 17

the method by which the p-value is calculated assumes the data adheres to a normal distribution

Question 18

What happens to the factors associated with relatively large p-values?

Answer 18

• difference in between group means decrease
• increase in standard deviation
• decrease in sample size

Question 19

What happens to the factors associated with relatively small p-values?

Answer 19

• difference in between group means increase
• decrease in standard deviation
• increase in sample size

Question 20

What is a one-tailed test entail?

Answer 20

Assessing the statistical significance of potential positive and negative differences between one groups and another

Question 21

What does a two-tailed test entail?

Answer 21

Assessing only the significance of a difference in one specific ‘direction’

Question 22

What is the main difference between one and two tailed test?

Answer 22

A one-tailed tests will only have one critical region whereas two-tailed tests will have two critical regions

Question 23

How to know when to use one-tailed or two-tailed test?

Answer 23

A one-tailed test looks for an “increase” or “decrease” in the parameter whereas a two-tailed test looks for a “change” (could be increase or decrease) in the parameter.

Question 24

What is the result of a two-tailed test differentiate between?

Answer 24

• any change (positive or negative)
• significant difference

Question 25

What is the result of a one-tailed test differentiate between?

Answer 25

• an increase or a decrease (a change in one direction)
• significant difference

Question 26

The number of tails evaluated in the analysis should also be reflected in the language used in the hypothesis.
Examples: One-tailed or Two-tailed?

(a) Does treatment with IL-5 have an impact on BAL eosinophil density during an asthma attack?
(b) Is novel drug ‘X’ a more effective treatment for symptoms of ADHD than methylphenidate?

Answer 26

a) two-tailed
b) one-tailed

Question 27

Parametric methods (e.g. t-test) generate p-values by…

Answer 27

…assuming a specific relationship between the variation of data points to the mean and the probability of observing them

Question 28

For non-parametric data,how do you generate p values?

Answer 28

A specific relationship doesn’t exist, therefore any test that assumes the relationship exists will generate an inaccurate p-value

Question 29

Mann-Whitney U test does…

Answer 29

…not assume any particular distribution and it is a non-parametric method.

Question 30

D