Week 5 Random Error

Question 1

What is random error?

Answer 1

= Error that results from data divergence, due to chance alone, of an observation on a sample of the true population

It can never be completely eliminated - but steps can be taken to reduce it.

Careful measurement of exposure and outcome and larger sample sizes

Question 2

What is the significance of Random Error?

Answer 2

In order to draw meaningful conclusions from study data, we need to be confident that the findings are not due to random error

We use statistical testing to estimate the probability that the results are due to random error (chance)

Question 3

How can we measure random error?

Answer 3

Hypothesis testing: Statistical tests that reflect the “statistical significance” of a finding. E.g. Creating a null hypothesis which we want to disprove, with a P interval of <0.05

Or through estimation of the range of values that are likely to contain the true value: Confidence intervals (e.g. the 95% convidence interval)

Question 4

Explain Hypothesis testing

Answer 4

When you perform a test of statistical significance you either reject or accept the null hypothesis. The null hypothesis is that there is not a significant difference between the two groups.
The alternative hypothesis is that there IS a statistical difference between the two groups.

The null hypothesis can only be rejected if there is significant statistical evidence: this is reflected by the P values.

P value regards a statistically significant result as being 0.05 then the null hypothesis cannot be rejected
If P=<0.05 then we can reject the null hypthesis because there is a significant difference

Question 5

What is Type I error?

Answer 5

Type I error = when the conclusion states there is a significant difference when there isn’t

I.e. the study would incorrectly reject the null hypothesis

Question 6

Type II error?

Answer 6

When the conclusion states that there is not a significant difference when there actually is one

I.e. the study would incorrectly accept the null hypothesis

Question 7

What is Alpha a?

Answer 7

The likelihood of producing a Type I error = Alpha(a) error.

It is expressed by the P value.

I.e. the p value is the level of Alpha(a) likelihood that we are willing to accept.

The P value may vary depending on the consequences of making a type I error

Question 8

What are the considerations for adopting a higher or lower P value?

Answer 8

The P value should be set lower for higher-risk studies.

E.g. if there is no known safe treatment for a severe disease, and you are trialling a treatment - may have more to gain by adopting a less stringent P value.

If there are already safe treatments available for a disease, and you are trialling a more dangerous but potentially more effective drug, then you should adopt a more stringent P value

Question 9

What is the relationship between Statistical and Clinical Significance

Answer 9

There is not necessarily one.

E.g. there may be found to be a statistically significant difference in performance in cognition tests after taking a certain drug. However, clinically, this may not actually affect patient outcomes. E.g. doing 0.8% better on a test may be statistically proven to be a real difference, but that ‘real’ difference may have no bearing on clinical outcomes

Question 10

What is the statistial approach to hypothesis tensting?

Answer 10

Test the null hypothesis that there is no significant difference

Null hypothesis is generally rejected if p=. And the P value is the likelihood of Alpha(a) - thus p value = likelihood of type I error

Question 11

What are the factors that influence P?

Answer 11

1) The magnitude of the main effect
The larger the difference between the groups, the more likely P is to be significant

2) The number of observations
Observations of larger numbers of participants are more likely to reap lower P values (statistically significant) than smaller study sizes

3) The spread in the data: the standard deviation
Wider distribution = higher P value (less significant)

Question 12

What are the shortcomings of P values?

Answer 12

They do not reflect systematic errors.

Although random error is important, and impossible to completely eliminate, the major concern in most studies is the presence of systematic errors

Question 13

What is systematic error?

Answer 13

Errors in the kinds of participants involved, measurement-taking biases? Observational errors (e.g. observed associations being confounded by other factors)

Question 14

What are estimations of true significance in a study results?

Answer 14

Point Estimates

Confidence Intervals

Question 15

What is a point estimate?

Answer 15

The effect size in the observed study

Effect size = a calculation (scale of 0-1) that can show whether or not a difference is meaningful

I.e. whether a statistical significance is CLINICALLY significant

The ‘effect size’ observed in a study - e.g. the RR, or OR

But it will always be higher or lower than the true effect because of random error

Question 16

Confidence Intervals

Answer 16

The range of values between 2 values within which you are (a given percentage) sure that the true value lies within

Because the point estimate always lies just above or below the true value, due to random error, the confidence interval gives you a range of values within which you can be (a given percentage) sure that the true value lies within

Question 17

E.g. of a point estimate and a CI

Answer 17

Point estimate of the weight loss involved in a diet program may be “7 kilos”. The CI, however, may range from 6.4-7.2 kilos.

If you calculated a 95% CI, you could state that, if you were to replicate that study again multiple times, the true interval would occur within the CI range 95% of the time.

Question 18

Interpretation of CIs

Answer 18

Narrower CIs indicate the more likely it is that the ‘main effect’ lies closer to the true value

Remember that with a 95%CI, the true value can still fall outside of the CI range 5 times out of 100

Question 19

What factors affect CIs?

Answer 19

1) The number ob observations:
larger sample sizes generally have narrower CI intervals, largely because larger sample sizes tend to have effect from random error

2) The Standard Deviation (spread in the data)
The less spread, the narrower the CI

Question 20

What is the relationship between CIs and Ps?

Answer 20

The wider the range of your CI, the less likely that your P value will be significant

E.g. if measuring weight loss… if the mean effect is 7 kilos, but the CI interval spans from -3 to 9kilos, the CI spans directly into the ‘no difference’ zone (of 0 or less weight loss). Thus, the P value would be >0.05

Question 21

What are the advantages of CIs over P values?

Answer 21

CIs convey more meaning to the reader

They provide informaiton about both the magnitude of the difference, and the uncertainty of the estimate - this helps us assess the statistical significance and the clinical or practical significance of the study

CIs can also reflect statistical power. Wide CIs may indicate that a relatively low statistica power may have been used