Research Skills 7 : Interpreting results

Question 1

What is Randomness

Answer 1

You should never ignore strange or interesting results

But be prepared that some apparent results may be due to random variation rather than a real effect.

Question 2

What does Random variation mean?

Answer 2

Random variation makes individual results highly unpredictable

But when you have a very large number of results, random variation makes the overall pattern behave in a mathematically predictable way

This is the basis of mathematical statistics

Question 3

What do Mathematical stats indicate ?

Answer 3

So mathematical statistics can only give us a very rough indication about how to interpret our data

But a rough indication is better than no indication!!

Question 4

Strong confidence in results

Answer 4

• If the effect size is large enough to be biologically significant
• If the effect size is large compared to variation
• Then it is very unlikely that random variation could account for the difference between the treated and control

Question 5

If no strong confidence in result..

Answer 5

If the effect size is not large compared to the variation

Then random variation in readings might account for the difference between the treated and control means

Question 6

Examples of Statistical difference

Answer 6

T-test
ANOVA - analysis of variance

these tests ask how does the variation compare with the effect size

Question 7

What happen if we take the mean of many observations

Answer 7

• We decrease the uncertainty in our results

- We can have confidence in results, even when the individual data points overlap

Question 8

How do SEM error bars help indicate uncertainty in results?

Answer 8

By providing a

• Simple
• Visual
• Intuitive
• If slightly rough way of evaluating statistical confidence

Question 9

What is SEM?

Answer 9

Standard error of the mean

• a measure of confidence in the experimental result
• As the number of observations gets larger, the s.e.m. gets smaller

But the s.e.m. gives an idea of the possible margin of error

Question 10

Caution of Error Bars

Answer 10

• Error bars should be treated as a rough indication because
  1. With low numbers of samples (<20) they underestimate the uncertainty
  2. Error bars are not additive
  3. All statistics give only a rough indication of confidence

Question 11

Confidence Intervals (C.I.s)

Answer 11

• There are mathematically more precise ways to ask how big any effect could be, or how small
• These get round the problem that s.e.m. error bars are not additive
• By using a t-test we can get the best estimate, even when we have a small number of observations

Question 12

How does C.I.s work?

Answer 12

• Using a statistics package like PRISM or SPSS

1. We enter the data for the treated
2. We enter the data for the control
3. We request a t-test on the data

Question 13

What does the T-test calculate?

Answer 13

calculates the 95% confidence interval for the difference between the means

Question 14

What is the 95% confidence interval?

Answer 14

Our best estimate for the effect size is the difference between the mean for treated and mean for controls

The confidence interval gives us an idea of how much larger or smaller

95% of the time, the real answer should be within this interval (but 5% of the time it will be outside)

Question 15

What are the Advantages of Confidence Intervals ?

Answer 15

• Combine numerical information on effect size, statistical confidence, and possible variation in the “real” effect size
• Ideal for simple comparisons such as treated vs control
• Now the preferred approach in clinical research and epidemiology

Question 16

What are the disadvantages of Confidence Intervals?

Answer 16

Harder to apply to more complex experiments, e.g. more than one control, more than one treatment