5. Introduction to key statistical Concepts

Question 1

What is the difference between population and sample?

Answer 1

•Population - all possible observations
of an experimental/study variable. This
is the population we are primarily
interested in.
•Sample - a selection of observations
taken from the population. Sample not
really of interest – we want to
generalise to the population.

Question 2

What is the difference between truth and observed considering coin toss?

Answer 2

The true probability of getting a “tail” on a fair
coin is a half, but what we observe may depart
from this by random variation

Question 3

Why is the observed value important?

Answer 3

Our observed value is our best estimate of the true or underlying tendency

Question 4

What is a hypothesis ??

Answer 4

A hypothesis is a statement that an underlying truth of scientific interest takes a particular
quantitative value, e.g.
– the prevalence of tuberculosis in a given population is 2 per 10,000 people – the coin is fair (i.e. probability of heads is 0.5)
– the new drug is neither better nor worse than the standard treatment (i.e. ratio of survival rates = 1.0)
– the new operation leads to neither more nor less post-operative pain (i.e. difference between pain scores = 0)

Question 5

Describe hypothesis testing

Answer 5

• Calculate the probability of getting an observation as

extreme as, or more extreme than, the one observed assuming that the hypothesis is true

Question 6

What can you conclude if the hypothesis testing probability is very small?

Answer 6

• If the probability is very small, it is reasonable to conclude
that the observation and the stated hypothesis are incompatible
• Therefore, with a small probability it is very unlikely that the
hypothesis is true
• This calculated probability is called the p-value

Question 7

What is the p value?

Answer 7

p-value - the probability of getting an observation as extreme as, or more extreme than, the one observed if the hypothesis is true.

Question 8

What is the interpretation of a p value <0.05?

Answer 8

– “Data inconsistent with the stated hypothesis”
– “Substantive evidence against the stated hypothesis”
– “Reasonable to reject the stated hypothesis”
– “Observations are statistically significantly different”

Question 9

What is the interpretation of p value >0.05?

Answer 9

– None of the above!
p-value ≥ 0.05” does NOT mean that the hypothesis has been proven, i.e. failing to reject the hypothesis does not mean that the hypothesis has been proven

Question 10

What are the limitations of hypothesis testing?

Answer 10

• Rejecting a hypothesis is not always useful:
– ‘p-value < 0.05’ is arbitrary: nothing special happens
between ‘p = 0.049’ and ‘p = 0.051’
– statistical significance depends on sample size:
flip a coin 3 times —>minimum p = 0.25 [i.e. 2 x 0.53]
– statistically significant ≠ clinically important
• However ‘p = 0.000001’ and ‘p = 0.6’ are often easy to interpret and p-values are used a lot