Lecture 17 - Resampling Statistics Part 2

Question 1

What should you do when using resampling techniques when there aren’t very many ways to shuffle the original sample?

Answer 1

It’s more efficient to use only the possible samples.

Question 2

What can bootstrap resamples be used to calculate?

Answer 2

The confidence interval of a mean

The standard error of a mean

Question 3

What’s the difference between a resampling test and a bootstrap resample?

Answer 3

A bootstrap resample resamples with replacement, meaning that one value from the original dataset can appear multiple times.

Question 4

What is the standard error of the mean when resampling?

Answer 4

The standard deviation of the means of all possible samples.

Question 5

How can the standard error of the mean be estimated using bootstraps?

Answer 5

The standard deviation of the bootstrapped means = SEM.

Question 6

Why do the SEM bootstrap estimate and the calculation using the SEM formula differ?

Answer 6

Because they’re both only estimates of the population SEM.

Question 7

How is the 95% confidence interval calculated with bootstraps?

Answer 7

It’s the middle 95% of the means.

Question 8

Using bootstrapping, how do you find the p value?

Answer 8

Simply the percentage of values above or below the value of interest.

Question 9

What is bootstrapping with a model fit?

Answer 9

Because comparing the mean to one value involves a very simple model of the world, bootstrapping can be generalised easily to more complex models than the mean - e.g. it is possible to do regression analysis and use bootstrapping on the values of the gradient.

Question 10

In calculating a confidence interval with bootstrapping, should resamples be made across or within experimental conditions?

Answer 10

Whichever suits the data best.

Question 11

When curve fitting with bootstraps, what do you do?

Answer 11

Resample appropriately, and for each resample fit the model curve again, obtaining new k and n values. Then analyse appropriately as in previous method.

Question 12

What are the pros of bootstrapping?

Answer 12

• It’s a very general method - any model can be used and any parameters investigated.
• Can be used to perform hypothesis testing (especially for one-sample tests).
• No assumptions, tables or equations.

Question 13

How many resamples should be generated with bootstraps?

Answer 13

1,000-10,000 depending on how accurate you want p.

Question 14

What does the necessary sample size for bootstrapping depend on?

Answer 14

Variance and bias.

Question 15

What are other resampling approaches, apart from normal resampling and bootstrapping?

Answer 15

Jack-knife and Monte-Carlo method.

Question 16

What does Jack-knifing involve?

Answer 16

It’s similar to bootstrapping, but rather than random sampling with replacement, resampling is done by the selection of all data except one.

Question 17

What is the Monte-Carlo method?

Answer 17

It involves creating data based on model simulations and then comparing these to real data. However, it only works in certain situations where good simulations are available.

Question 18

What software can be used for resampling?

Answer 18

• S-plus / R - David Howell, very basic.
• Python, Matlab or equivalent scripting language.
• C/C++