Levels, sample size and power Flashcards
what is data collected at?
- data collected at different levels
what do levels of data analysis depend on?
- depends on design and measures
what are the two different levels of data analysis?
- within a person
- between people
what are four examples of data within a person?
- response on questionnaire
- reaction time
- blood concentration
- physiological measurement
what are four examples of data analysis between people?
- individuals in a sample
- samples in a population
- classes/ cohorts in a school
- districts in a city
what is the most common level of data analysis?
- between people is most common
how many repetitions should you use?
- no hard rule
- more is better
what are the limitations of large sample sizes?
- time, money and resources limited
- participants’ effort, endurance, boredom, will- power and kindness are limited
- benefit increases only with square root of N
how do you double statistical power?
- quadruple number of repetitions
what do you do with all the repetitions?
- take averages
- one per design level
what is the alternative with all the averages?
- take average of differences
what do you do with all the averages of averages?
- analyse them
- use t- test, correlation, ANOVA, GLM
how many levels of data analysis is there?
- three
1-3
what is level 1 of data analysis? - give an example
- within- person, within- measurement
e.g., using 100 individual heart beats to estimate the heart rate
what type of data is level 1?
- raw data
what is level 2 data analysis?- give an example
- within- person, within condition
e.g., using 10 repetitions of each condition to estimate pre/ post HRs
what data/ statistic is level 2?
- summary data
- descriptive statistics
what is level 3 data analysis? - give an example
- between people
e.g., using 16 athletes in each group to estimate intervention effect
what statistics is level 3?
- inferential statistics
what do all general linear model statistical tests assume?
- assumes your data are independently sampled
how is the assumption not true for a heart rate experiment?
- heart rate now is not independent from your HR 5 seconds ago
what happens to the measurements from the same individual?
- correlated or dependent
what should you use for a valid GLM test?
- only use one value per person and condition
how can you use all the level 1 data?
- use a multi- level mixed model regression (MLMM)
what designs usually need less data?
- within- subject (repeated) designs
what is statistical power?
- the probability that you will find a significant result
what does statistical power increase with?
- increases with the square root of N
what do you need to set in statistics?
- set an arbitrary level of significance
how do you evaluate sample size?
- evaluate sample size in context of other things
do you need more than 20 for parametric GLM stats?
- NO, false assumption
- t-tests were created for N= 4 to 10
- more is usually better
what should you do if the study has a small sample size?
- poor criticism
- if you know it is too small just tell me what size it should be
what is assumed about the effect when there is a significant result?
- assuming that there is a real effect
- whatever your hypothesis, you assume it is exactly true
what is assumed about the sample when finding a significant result?
- that it is exactly as big as you say it is
what do you need to specify? what is the equation?
- specify an effect size> R2, f, cohen’s d
Cohen’s d = t / sqrt (N)
what does statistical power assume about other statistical assumptions?
- assuming that other statistical assumptions are true
- independent sampling
- similar variance, independent residuals, etc
what is the simplest case when calculating the effect size?
- a t- test
how do you guess the sample size?
- independent samples t-test
- calculate N required
- calculate Cohen’s d
are there many hard rules in statistics?
- no
- very few hard rules
what is sample size not easy to do?
- not easy to assess or criticise
how many datapoints should be used?
- more datapoints is usually better
how can sample size be analysed?
- re effect size
what can power analysis help with?
- helps you design a study