Data analysis: level, sample size, power (wk 7)

Question 1

What are levels of data analysis and within a person/ between people?

Answer 1

-Levels of data analysis -> Data are usually collected at different levels.
1. Within a person -> Response on question(naire), reaction time, blood concentration, physiological measurement
2. Between people -> Individuals in a sample, samples in a population, classes/ cohorts in a school, districts in a city

Question 2

How many repetitions should you do?

Answer 2

There is no hard ‘rule’; in general: more is better but: Time, money and resources are limited, Participants’ effort, endurance, boredom, will-power, kindness are limited and Statistical benefit increases only with square root of N – to double your statistical power, you need to quadruple the number of repetitions.

Question 3

What to do with all the repetitions?

Answer 3

In general take averages, one per design level. Alternatively, take average of differences.

Question 4

What do I do with all the averages?

Answer 4

Analyse them – with t-test, correlation, ANOVA, GLM. Mixed ANOVA: Athlete (speed, endurance) x Time (pre, post)

Question 5

What is level 1, 2 and 3 of data analysis?

Answer 5

Level 1 -> Within-person, within-measurement (raw data) e.g. using 100 individual heart beats to estimate the HR
Level 2 -> Within-person, within-condition (summary data/ descriptive statistics) e.g. using 10 repetitions off each condition to estimate pre/post HRs
Level 3 -> Between-people (inferential statistics) e.g. using 16 athletes in each group to estimate intervention effect

Question 6

How to use level 1 data

Answer 6

All GLM statistical tests assume your data are independently sampled. Measurements from the same individual are correlated or dependent. For a valid GLM test, only use one value per person and condition. You need a multi-level mixed model regression (MLMM).

Question 7

What are the general principles of data?

Answer 7

Within-subject (repeated) designs usually need less data. Statistical power increases with the square root of N. You need to set an arbitrary level of ‘significance’. You need to evaluate sample size in context of other things.

Question 8

Assumptions of statistical power:

Answer 8

-Statistical power -> Is the probability that you will find a significant result. Assuming:
* There is a real effect -> Whatever your hypothesis, you assume it is exactly true
* That it is exactly as big as you say it is -> You need to specify an effect size – R2, f, Cohen’s d. We will just look at t-tests, Cohen’s d = t/ sqrt(N).
* The other statistical assumptions are true -> Independent sampling. Similar variance, independent residuals etc.