WEEK 2 Flashcards
Treatment exposure effects RR
If trial has 1000 times as many participants but RR still the same
Believe the larger trial more because of bigger sample size
Uncertainty
Some manner of unpredictability
In any study the treatment/exposure effect is calculated using the participants in the study
No study recruits every single eligible participant
Effect from sample of participants in study is only an estimate of the effect in the entire population
Leads us to sampling variation
Sampling
We want to know about population of people
May not be feasible to make observations on entire population
Use a sample of participants
Expect inferences from sample to be similar to population
Difference between truth of population and the sample is called sample variability
Sample is only selection of entire population
Different samples from same population will give different results
This is due to sampling variation or sampling error
Each sample that could be taken may have a different mean
Sampling variation
A parameter is a value that refers to the population that we cannot know
Its a singular value as there’s only one population
Can only estimate a parameter using statistics
A statistic may be a value that refers to the sample that we can calculate
A statistic can change from sample to sample
The sample may not be exact representation of the population
Sampling variation application
Sampling variability is the amount of difference between the statistic calculated from the sample and the parameter value from the population
If the variability is low then there is small difference between the sample statistic and population parameter
If the variability is high there is a large difference between the sample statistic and population parameter
Sampling distribution
If you keep taking samples and calculating their mean
Plot a histogram of the mean from each sample- sampling distribution
-with a large number of samples it approximates the normal distribution
The spread in this distribution captures the variability that we could get in the mean in one of our samples
The standard error describes how far, on average, a sample mean is from population value
Standard error- variability in mean
If taking multiple samples, the mean value in each sample may differ
The standard error describes the variability in the means
It tells us how accurate the mean of any particular sample is compared to the true population mean
A large standard error would indicate the mean from each sample are likely to differ a lot and so could be an inaccurate representation of true population mean
The mean of 95% of samples is within 2 SEs of population mean
Standard deviation- variability in sample
In a sample the standard deviation explains how far on average a measurement is from the mean
The larger the standard deviation the more variation exists in our observations describes variability in our sample
About 95% of the sample observation within 2 SDs of the sample mean
Larger studies are better
Results of repeated studies are not expected to be exactly the same
-the results of small studies can be quite some distance from the truth
-the bigger the study the more likely that its results will be close to the truth
The standard error describes how precisely we have estimated the treatment effect
-small study- larger standard error- we are uncertain
-large study- small standard error- we are more certain
SD same but SE is smaller in larger study
Standard deviation vs standard error
The standard deviation- describes study data. 95% observations in the sample are in this interval
The standard error- about precision of estimates. 95% of studies would have a mean in this interval
Expressing uncertainty
Because our statistic is calculated from one samples using a point estimate alone to describe statistic can be uninformative
When expressing an estimate of a population parameter (eg odds ratio, risk ratio, mean difference, risk difference) its good practice to include confidence interval alongside it
The confidence interval communicates how accurate our estimate is likely to be
Confidence intervals
Standard errors are not easily interpreted
So want to present range of likely values
Use confidence intervals to describe the uncertainty in each estimate
We can calculate confidence intervals using properties of normal distribution
We cannot take lots of samples or measure whole population so take one sample and calculate mean
Use a confidence interval to express the range of values which we are pretty sure the population parameter will lie in
This range of values is calculated from what we have observed. It’s centre is the sample mean and we have room either side for our uncertainty
The size of confidence interval depends on
Variation within the population
Size of the sample
How confident we want to be
Population variation
If all values in the population are the same/similar then our sample will follow the same pattern
-any sample of patients will be similar to another sample
-an estimate of the parameter (eg OR, RR) will be pretty close to the true population value
If values in the population differ our sample will follow the same pattern
-any sample of patients is likely to differ from another sample
-less sure that parameter estimate is close to the true population value
Greater variation in population= wider confidence interval
Sample size
If the size of our sample is small:
-we have little information to base our inference on
-any sample of patients is likely to differ from another sample
If the size of sample is large:
-more information to base our inference on
-impact of outliers/unusual values are evened out in the sample
-any sample of patients is likely to be similar to another sample
Small sample size= wider confidence interval
Confidence level
How confident do we want to be about our inferences
To be 100% accurate the confidence interval would have to be form minus infinity to infinity
Trade off accuracy for a more informative range
When describing 95% CI:
-our confidence is that the procedure of calculating the CI means we would not include the true population mean 5% of the time
-95% of the time it would contain true mean
Confidence interval for mean
To calculate lower and upper values for our confidence intervals
The lower value is 2 SEs less than mean difference
The upper value 2 SEs above the mean difference
Confidence intervals for exposure effect
Lower value is 2 SEs less than sample exposure effect
Upper value is 2 SEs more than sample exposure effect
Confidence interval interpretation RD
An absolute difference of 0 essentially means no difference between 2 comparison groups
RD or MD
If the 95% CI for a RD is above 0 (the null value) then we can be 95% certain the the risk in the treatment arm is greater than that in control arm
If the 95% CI for RD below 0 (the null value) then we can be 95% certain that the risks in treatment arm is less than that in control arm
If the 95% CI for RD contains 0(the null value) then there’s not enough evidence to say the risk is different in treatment arm to control arm
Confidence interval interpretation RR and OR
A relative difference of 1 essentially means no difference between 2 comparison groups
If the 95% CI for RR is above 1 (nulls value) then we can be 95% certain that risk in treatment arm is greater than control
If 95% CI RR below 1we can be 95% certain that risk in treatment arm is less than that in control arm
If 95% CI RR contains 1 then there’s not enough evidence to say the risk is different in treatment arm to control arm
Constructing a hypothesis test
Specify the null and alternative hypothesis
Assume the null hypothesis is true and calculate test statistic
Convert to p-value
Assess the evidence
Interpret the result
The null and alternative hypothesis
A null hypothesis H0 and an alternative hypothesis H1 or HA are proposed
The null hypothesis is that there’s no difference
The alternative hypothesis is that there is a difference
Assuming null hypothesis is true
We assume the null hypothesis is true
We measure how close the observed data is to what we would expect if the null hypothesis is true
How we measure this depends on test being performed
The p value
The p value is the probability that the data (or more extreme data) could have arisen if null hypothesis is true
What is the chance we could end up with these results (or more extreme) if the null hypothesis were true
P value interpretation guide
P>-0.1 little or no evidence against H0
0.05-<p<0.1 little evidence against H0
0.01-<p<0.05 evidence against H0
0.001-<p<0.01 strong evidence against H0
P<0.001 very strong evidence against H0
Statistical significance
A result is said to be statistically significant if the p value reflects a Lower probability of seeing the result (or more extreme) by chance than the level we have set for statistical significance
The levels we use reflects the probability we are willing to incorrectly say a result is statistically significant
Absence of evidence is not evidence of absence
P values indicate how much evidence we have against null hypothesis
A result may indicate little or no evidence that 2 groups are different- this does not mean two groups are the same
Assessing evidence
If p value< 0.001
The probability of seeing the result by chance is <0.001. This results is significant at the 5% level
Very strong evidence against H0
P values and confidence intervals
Statistical significance:
-assessed by p value
-shows whether an effect exists
-no comment on size of effect
Clinical importance:
-assessed using estimates and confidence intervals (tells us about the uncertainty of our estimates)
-judges whether effects are large enough to be clinically important (need clinical expertise)
Cis and p values for treatment effects
If 95% CI contains null value then P>0.05
If 95% CI doesnt contains null value p<0.05
If 95% CI ends at null value p=0.05
Comparing CIs
Significant difference no overlap
Unclear if significantly different- need statistical test - when CI limits overlap but point estimates in own CI
Not significantly different: overlap point estimates
Synthesising evidence from multiple studies
Systematic reviews synthesise evidence from multiple studies
-studies addressing the same clinical question
-in similar populations
The technique of meta analysis is used to pool data over the studies to come up with a combined (pooled) treatment effect
Synthesising over studies increases the effective sample size, decreases uncertainty, results in tighter CIs
Forest plot is a useful way of summarising results of a systematics review
Consists of a graphical display of treatment effects (and CIs) from each study along with pooled estimate
There are various methods used to pool estimates from the individual study but all:
-give greater weight to larger studies
-give smaller weight to small studies
Problems with statistical significance
Practices that reduce data analysis or scientific inference to mechanical “bright line” rules such as p<0.05 for justifying scientific claims or conclusions can lead to erroneous beliefs and poor decision making
Dichotomania and p=0.05
P<0.05 has been used as a cut of for statistical significance
When p is less than 0.05 this is often over interpreted as ‘an effect definitely exists’ sometimes abbreviated p values are given such as p<0.05 or NS reinforcing dichotomania
Loss of information regarding strength of evidence: p=0.06 is very different to p=0.9 and very similar to p=0.04
Nerve conclude there is no difference or no association just because p value is larger then a threshold like 0.05 or equivalently because CIs includes 0
Neither should we conclude that 2 studies conflict because one had a statistically significant result and the other did not
Solution- estimates and uncertainty
Must learn to embrace uncertainty
Focus on CIs
Describe practical implications of all values inside the interval especially the observed effect and limits
Remember all values between intervals limits are reasonably compatible with the data given statistical assumptions used to compute the interval
Doesn’t mean values outside interval are incompatible they are just less compatible.
Discuss the point estimate
Not all values inside are equally compatible with the data given the assumptions
The point estimate is the most compatible and values near it are more compatible than those near the limits
Interpreting point estimate while acknowledging its uncertainty will keep you from making false declarations of ‘no difference’ and from making overconfident claims
95% arbitrary choice
Third, like the 0.05 threshold from which it
came, the default 95% used to compute
intervals is itself an arbitrary convention. A different level can be justified, depending
on the application.
Interval estimates can perpetuate the
problems of statistical significance when the
dichotomization they impose is treated as a
scientific standard.
