Lecture 16 - Resampling Statistics Part 1 Flashcards
In what ways do resampling techniques differ from more traditional statistical methods?
Most statistical tests are based on ‘closed form’ equations based in the 19th and early 20th C, and are consequently based on a particular model of the world. They therefore tend to make a lot of assumptions (those that don’t have little power), whereas resampling techniques are assumption-free(er) whilst retaining power.
Why are resampling techniques better than traditional tests?
They: Make fewer assumptions Are very general (and can therefore be used for a variety of conditions) Have no equations or tables Force us to think about the data
Why are resampling techniques not more popular?
They are relatively new
Parametric stats do a reasonably good job and are what people were taught
Resampling requires some programming (not available in SPSS)
People think they’re complex because they’re new
People don’t like actually thinking about their data
What is the motivation for inferential statistics (taking meaning from data e.g. ANOVA)?
To determine the probability that the differences measured were caused by sampling error.
What do resampling techniques do?
Determine the likely sampling error by repeating the sampling process many times and looking at the variability in the resampling.
What are between-subject randomisation tests also known as?
Permutation tests
What is the process involved in a between-subject randomisation test?
Aim to determine the likelihood of getting such differences if the data came from a single population, therefore simulate this resampling from one population in order to check what range of differences is normal.
In practice, shuffle the values, randomly assigning them to groups, and count how often the difference is larger than the one observed - this percentage is the p value.
How else, apart from comparing means, can a between-subject randomisation test be used?
To compare sds, or any other value really, meaning that a new test is not necessary if you change your mind about what you want to know from the data.
What is the process involved in a within-subject randomisation test?
The data values are reshuffled for each subject, rather than across the whole dataset, essentially randomising the sign of the difference for each pair.
Then the number of resampled mean differences that were more extreme (either direction) become p.
How is n accounted for in resampling tests?
The sample size for the resamples has to be the same as the original data and so the variance in the mean differences will automatically reflect the number of participants. However it does require a large and unbiased sample!
As sample size increases, what changes about normal distribution curves of randomisation tests?
Variance decreases and consequently they look taller and thinner.
