EAB - Estimation and Significance Tests and P Values

Question 1

What is sampling error?

Answer 1

Samples provide an incomplete picture of the population.

Different samples will give different estimates, which is called ‘sampling error’.

Question 2

What is sampling distribution?

Answer 2

Sample estimates (e.g. means) are calculated from multiple samples from the same population.

They will will then have a distribution of differing values which is known as the ‘sampling distribution’.

Question 3

What are two measures we can introduce to deal with uncertainty in drawing conclusions?

Answer 3

• Confidence interval:
  If we are estimating some quantity from our data, for example, the proportion of patients who have a particular attribute, then we can quantify the imprecision in the estimate using a confidence interval.
• Statistical significance test:
  If we are testing a hypothesis, for example, comparing blood pressure in two groups, then we can do a statistical significance test which helps us to weigh the evidence that the sample difference we have observed is in fact a real difference.

Question 4

What is the relationship between sample size and how close it is to the true mean?

Answer 4

The bigger the sample size, the closer the estimate is to the true mean.

Question 5

What is the relationship between spread of data and how close it is to the true mean?

Answer 5

The smaller the spread of data (standard deviation), the closer the estimate is to the true mean.

Question 6

What is a standard error?

Answer 6

A standard error (SE) is an indication of the extent of the sampling error.

Standard error tells us how much a sample mean tends to vary from the population mean (true mean). It provides an estimate of the precision of the sample mean.

Question 7

How do you calculate standard error?

Answer 7

For a sample mean, it can be calculated from the standard deviation divided by the square root of the sample size.

(SE = SD / √[𝑁])

Question 8

How can standard error be used to calculate a confidence interval?

Answer 8

The true (population) mean can be expected to lie in the range: (sample mean – 1.96 standard errors) to (sample mean + 1.96 standard errors) in 95% of calculations.

Question 9

What are our assumptions when calculating a 95% confidence interval from population mean?

Answer 9

• this is normal data or a large sample (at least 60)
• the sample is chosen at random from the population
• the observations are independent of each other

Question 10

What are our assumptions when calculating a 95% confidence interval from population proportion?

Answer 10

• the sample is chosen at random from the population
• the observations are independent of each other
• the proportion with the characteristic is not close to 0 or 1
• np and n(1-p) are each greater than 5 (large sample)

Question 11

How do you calculate the standard error for proportion?

Answer 11

Multiply the proportion with the characteristic by the proportion without the characteristic:
p(1-p)

Divide by the sample size:
p(1-p)/n

Take the square root to deduce the SE:
√[(𝑝 × (1 − 𝑝)/𝑛)]

Question 12

What is a significance test (and its benefit)?

Answer 12

A significance test uses data from a sample to show the likelihood that a hypothesis about a
population is true. There are always two mutually exclusive hypotheses since, if the hypothesis being tested is not true, then the opposite hypothesis must be true.

A measure of the evidence for or against the hypothesis is provided by a P value.

Question 13

What is the null hypothesis?

Answer 13

The null hypothesis is the baseline hypothesis which is usually of the form ‘there is no difference’ or
‘there is no association’.

The corresponding alternative hypothesis is ‘there is a difference’ or ‘there is an association’.

Question 14

What is a two-sided test (two-tailed test)?

Answer 14

It is known as a two-sided or two-tailed test when the alternative hypothesis is general and allows the difference to be in either
direction.

Question 15

What is a one-sided test (one-tailed test)?

Answer 15

It is known as a one-sided or one-tailed test when the alternative hypothesis is not general and allows the difference to be in only one
direction.

Two-sided tests should always be used unless there is clear justification at the outset to use a one-sided test.

Question 16

What are the steps in doing a significance test?

Answer 16

1. Specify the hypothesis of interest as a null and alternative hypothesis.
2. Decide what statistical test is appropriate.
3. Use the test to calculate the P value.
4. Weigh the evidence from the P value in favour of the null or alternative hypothesis.

Question 17

Describe the types of errors in significance testing.

Answer 17

Since a significance test uses sample data to make inferences about populations, using the results from a sample may lead to wrong conclusion:

TYPE 1 ERROR:
this is getting a significant result in a sample when the null hypothesis is in fact true in the underlying population (‘false significant’ result).

We usually set a limit of 0.05 (5%) for the probability of a type 1 error which is equivalent to a 0.05 cut-off for statistical significance.

TYPE 2 ERROR:
this is getting a non-significant result in a sample when the null hypothesis is in fact
false in the underlying population (‘false non-significant’ result).

It is widely accepted that the probability of a type 2 error should be no more than 0.20 (20%).

Question 18

Describe what a P value is.

Answer 18

A P value is a probability, and therefore lies between 0 and 1. It comes from a statistical test that is testing a particular null hypothesis.

It expresses the weight of evidence in favour of or against the stated null hypothesis.

Precise definition: P value is the probability, given that the null hypothesis is true, of obtaining data as extreme or more extreme than that observed.

Question 19

What is the cut off point for a p value, and what does that indicate?

Answer 19

0.05 or 5% is commonly used as a cut-off, such that if the observed P is less than this (P<0.05) we consider that there is good evidence that the null hypothesis is not true. This is directly related to the type 1 error rate.

If 0.05 is the cut-off then P< 0.05 is commonly described as statistically significant and P≥0.05 is described as not statistically significant.

Question 20

List some factors that affects the size of the p value.

Answer 20

• the size of the real effect in the population sampled
• the sample size
• the variability of the measure involved

Question 21

What does clinical significance indicate?

Answer 21

This indicates that the difference observed is large enough to be clinically meaningful. It is not necessarily related to statistical significance as it is a clinical judgement and not a mathematical
quantity.