WEEK 2

Question 1

Treatment exposure effects RR

Answer 1

If trial has 1000 times as many participants but RR still the same
Believe the larger trial more because of bigger sample size

Question 2

Uncertainty

Answer 2

Some manner of unpredictability
In any study the treatment/exposure effect is calculated using the participants in the study
No study recruits every single eligible participant
Effect from sample of participants in study is only an estimate of the effect in the entire population
Leads us to sampling variation

Question 3

Sampling

Answer 3

We want to know about population of people
May not be feasible to make observations on entire population
Use a sample of participants
Expect inferences from sample to be similar to population
Difference between truth of population and the sample is called sample variability
Sample is only selection of entire population
Different samples from same population will give different results
This is due to sampling variation or sampling error
Each sample that could be taken may have a different mean

Question 4

Sampling variation

Answer 4

A parameter is a value that refers to the population that we cannot know
Its a singular value as there’s only one population
Can only estimate a parameter using statistics
A statistic may be a value that refers to the sample that we can calculate
A statistic can change from sample to sample
The sample may not be exact representation of the population

Question 5

Sampling variation application

Answer 5

Sampling variability is the amount of difference between the statistic calculated from the sample and the parameter value from the population
If the variability is low then there is small difference between the sample statistic and population parameter
If the variability is high there is a large difference between the sample statistic and population parameter

Question 6

Sampling distribution

Answer 6

If you keep taking samples and calculating their mean
Plot a histogram of the mean from each sample- sampling distribution
-with a large number of samples it approximates the normal distribution
The spread in this distribution captures the variability that we could get in the mean in one of our samples
The standard error describes how far, on average, a sample mean is from population value

Question 7

Standard error- variability in mean

Answer 7

If taking multiple samples, the mean value in each sample may differ
The standard error describes the variability in the means
It tells us how accurate the mean of any particular sample is compared to the true population mean
A large standard error would indicate the mean from each sample are likely to differ a lot and so could be an inaccurate representation of true population mean
The mean of 95% of samples is within 2 SEs of population mean

Question 8

Standard deviation- variability in sample

Answer 8

In a sample the standard deviation explains how far on average a measurement is from the mean
The larger the standard deviation the more variation exists in our observations describes variability in our sample
About 95% of the sample observation within 2 SDs of the sample mean

Question 9

Larger studies are better

Answer 9

Results of repeated studies are not expected to be exactly the same
-the results of small studies can be quite some distance from the truth
-the bigger the study the more likely that its results will be close to the truth
The standard error describes how precisely we have estimated the treatment effect
-small study- larger standard error- we are uncertain
-large study- small standard error- we are more certain
SD same but SE is smaller in larger study

Question 10

Standard deviation vs standard error

Answer 10

The standard deviation- describes study data. 95% observations in the sample are in this interval
The standard error- about precision of estimates. 95% of studies would have a mean in this interval

Question 11

Expressing uncertainty

Answer 11

Because our statistic is calculated from one samples using a point estimate alone to describe statistic can be uninformative
When expressing an estimate of a population parameter (eg odds ratio, risk ratio, mean difference, risk difference) its good practice to include confidence interval alongside it
The confidence interval communicates how accurate our estimate is likely to be

Question 12

Confidence intervals

Answer 12

Standard errors are not easily interpreted
So want to present range of likely values
Use confidence intervals to describe the uncertainty in each estimate
We can calculate confidence intervals using properties of normal distribution
We cannot take lots of samples or measure whole population so take one sample and calculate mean
Use a confidence interval to express the range of values which we are pretty sure the population parameter will lie in
This range of values is calculated from what we have observed. It’s centre is the sample mean and we have room either side for our uncertainty

Question 13

The size of confidence interval depends on

Answer 13

Variation within the population
Size of the sample
How confident we want to be

Question 14

Population variation

Answer 14

If all values in the population are the same/similar then our sample will follow the same pattern
-any sample of patients will be similar to another sample
-an estimate of the parameter (eg OR, RR) will be pretty close to the true population value
If values in the population differ our sample will follow the same pattern
-any sample of patients is likely to differ from another sample
-less sure that parameter estimate is close to the true population value
Greater variation in population= wider confidence interval

Question 15

Sample size

Answer 15

If the size of our sample is small:
-we have little information to base our inference on
-any sample of patients is likely to differ from another sample
If the size of sample is large:
-more information to base our inference on
-impact of outliers/unusual values are evened out in the sample
-any sample of patients is likely to be similar to another sample
Small sample size= wider confidence interval

Question 16

Confidence level

Answer 16

How confident do we want to be about our inferences
To be 100% accurate the confidence interval would have to be form minus infinity to infinity
Trade off accuracy for a more informative range
When describing 95% CI:
-our confidence is that the procedure of calculating the CI means we would not include the true population mean 5% of the time
-95% of the time it would contain true mean

Question 17

Confidence interval for mean

Answer 17

To calculate lower and upper values for our confidence intervals
The lower value is 2 SEs less than mean difference
The upper value 2 SEs above the mean difference

Question 18

Confidence intervals for exposure effect

Answer 18

Lower value is 2 SEs less than sample exposure effect
Upper value is 2 SEs more than sample exposure effect

Question 19

Confidence interval interpretation RD

Answer 19

An absolute difference of 0 essentially means no difference between 2 comparison groups
RD or MD
If the 95% CI for a RD is above 0 (the null value) then we can be 95% certain the the risk in the treatment arm is greater than that in control arm
If the 95% CI for RD below 0 (the null value) then we can be 95% certain that the risks in treatment arm is less than that in control arm
If the 95% CI for RD contains 0(the null value) then there’s not enough evidence to say the risk is different in treatment arm to control arm

Question 20

Confidence interval interpretation RR and OR

Answer 20

A relative difference of 1 essentially means no difference between 2 comparison groups
If the 95% CI for RR is above 1 (nulls value) then we can be 95% certain that risk in treatment arm is greater than control
If 95% CI RR below 1we can be 95% certain that risk in treatment arm is less than that in control arm
If 95% CI RR contains 1 then there’s not enough evidence to say the risk is different in treatment arm to control arm

Question 21

Constructing a hypothesis test

Answer 21

Specify the null and alternative hypothesis
Assume the null hypothesis is true and calculate test statistic
Convert to p-value
Assess the evidence
Interpret the result

Question 22

The null and alternative hypothesis

Answer 22

A null hypothesis H0 and an alternative hypothesis H1 or HA are proposed
The null hypothesis is that there’s no difference
The alternative hypothesis is that there is a difference

Question 23

Assuming null hypothesis is true

Answer 23

We assume the null hypothesis is true
We measure how close the observed data is to what we would expect if the null hypothesis is true
How we measure this depends on test being performed

Question 24

The p value

Answer 24

The p value is the probability that the data (or more extreme data) could have arisen if null hypothesis is true
What is the chance we could end up with these results (or more extreme) if the null hypothesis were true

Question 25

P value interpretation guide

Answer 25

P>-0.1 little or no evidence against H0
0.05-<p<0.1 little evidence against H0
0.01-<p<0.05 evidence against H0
0.001-<p<0.01 strong evidence against H0
P<0.001 very strong evidence against H0

Question 26

Statistical significance

Answer 26

A result is said to be statistically significant if the p value reflects a Lower probability of seeing the result (or more extreme) by chance than the level we have set for statistical significance
The levels we use reflects the probability we are willing to incorrectly say a result is statistically significant

Question 27

Absence of evidence is not evidence of absence

Answer 27

P values indicate how much evidence we have against null hypothesis
A result may indicate little or no evidence that 2 groups are different- this does not mean two groups are the same

Question 28

Assessing evidence

Answer 28

If p value< 0.001
The probability of seeing the result by chance is <0.001. This results is significant at the 5% level
Very strong evidence against H0

Question 29

P values and confidence intervals

Answer 29

Statistical significance:
-assessed by p value
-shows whether an effect exists
-no comment on size of effect
Clinical importance:
-assessed using estimates and confidence intervals (tells us about the uncertainty of our estimates)
-judges whether effects are large enough to be clinically important (need clinical expertise)

Question 30

Cis and p values for treatment effects

Answer 30

If 95% CI contains null value then P>0.05
If 95% CI doesnt contains null value p<0.05
If 95% CI ends at null value p=0.05

Question 31

Comparing CIs

Answer 31

Significant difference no overlap
Unclear if significantly different- need statistical test - when CI limits overlap but point estimates in own CI
Not significantly different: overlap point estimates

Question 32

Synthesising evidence from multiple studies

Answer 32

Systematic reviews synthesise evidence from multiple studies
-studies addressing the same clinical question
-in similar populations
The technique of meta analysis is used to pool data over the studies to come up with a combined (pooled) treatment effect
Synthesising over studies increases the effective sample size, decreases uncertainty, results in tighter CIs
Forest plot is a useful way of summarising results of a systematics review
Consists of a graphical display of treatment effects (and CIs) from each study along with pooled estimate
There are various methods used to pool estimates from the individual study but all:
-give greater weight to larger studies
-give smaller weight to small studies

Question 33

Problems with statistical significance

Answer 33

Practices that reduce data analysis or scientific inference to mechanical “bright line” rules such as p<0.05 for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decision making

Question 34

Dichotomania and p=0.05

Answer 34

P<0.05 has been used as a cut of for statistical significance
When p is less than 0.05 this is often over interpreted as ‘an effect definitely exists’ sometimes abbreviated p values are given such as p<0.05 or NS reinforcing dichotomania
Loss of information regarding strength of evidence: p=0.06 is very different to p=0.9 and very similar to p=0.04
Nerve conclude there is no difference or no association just because p value is larger then a threshold like 0.05 or equivalently because CIs includes 0
Neither should we conclude that 2 studies conflict because one had a statistically significant result and the other did not

Question 35

Solution- estimates and uncertainty

Answer 35

Must learn to embrace uncertainty
Focus on CIs
Describe practical implications of all values inside the interval especially the observed effect and limits
Remember all values between intervals limits are reasonably compatible with the data given statistical assumptions used to compute the interval
Doesn’t mean values outside interval are incompatible they are just less compatible.

Question 36

Discuss the point estimate

Answer 36

Not all values inside are equally compatible with the data given the assumptions
The point estimate is the most compatible and values near it are more compatible than those near the limits
Interpreting point estimate while acknowledging its uncertainty will keep you from making false declarations of ‘no difference’ and from making overconfident claims

Question 37

95% arbitrary choice

Answer 37

Third, like the 0.05 threshold from which it
came, the default 95% used to compute
intervals is itself an arbitrary convention. A different level can be justified, depending
on the application.
Interval estimates can perpetuate the
problems of statistical significance when the
dichotomization they impose is treated as a
scientific standard.