Brainscape's Knowledge GenomeTM


Top Flashcards (Certified)
All Flashcards

research methods 2 > Effect Size > Flashcards

Effect Size Flashcards

1
Q

why is effect size useful?

A

experiments intend to find the effect of your IV on a DV

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

what does a p value tell us?

A

measures the probability of obtaining the observed results, assuming that the null hypothesis is true
- tells us whether an effect exists

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

what does the p value not tell us?

A

the size of the effect

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

what does effect size measure?

A

indicates the proportion of the variance explained

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

what effect size is used for correlation and regressions?

A

R and R^2

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

what effect size is used for T-tests?

A

Cohen’s d

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

what effect size is used for one way ANOVAs?

A

Eta Squared

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

what effect size is used for factorial anovas (where the effect is made up of different variables)?

A

Parietal Eta Squared

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

how do you calculate cohen’s d?

A

differences between the mean as a function of (/divided by) the standard deviation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

what is Cohen’s d?

A

standardised score representing the difference between the group mean

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

d = m1 - m2 / 𝜎

A

D is the difference in means -> scaled by the standard deviation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

what is cohen’s d an example of?

A

a standardised score (conducted with means rather than individual scores)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

what are the benefits when we scale by standard deviation in cohen’s d?

A
  • D does not dependent on a sample size (you can compare cohen’s d for a small pilot study and the full experiment because it’s standardised and divided by the standard deviation)
  • allows you to compare studies effect with other studies in literature
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

what is cohen’s d for ANOVA (where there are multiple groups)?

A

difference between largest and smallest group mean scaled by standard deviation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

cohen’s d for anova’s calculations

A

d = mean(max) - mean (min) / 𝜎

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

what are the assumptions for cohen’s d for ANOVAs calculation?

A
  • standard deviation is assumed to be constant across groups
  • only true for samples with met ANOVA assumptions (homogeneity of variance assumption -> assessed using Levene’s test)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

There are various conventions to calculate the overall standard deviation for an ANOVA. What are these?

A
  • averaging group SD
  • taking the smaller SD (more conversation)
  • pooling the variance (looking at different combinations and using the outcome of that)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

what is eta squared?

A

proportion of variance explained by your experiment
(variance that’s explained by your variance / total variance of the model)
- used for one-way ANOVAs

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

why is eta squared only used for one way ANOVAs?

A

cause only one variable and one effect size

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

what is the equation for eta squared?

A

Eta(n)2 = SSeffect / SStotal

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

how to calculate eta squared?

A

divide the sum of squares for the effect by the total sum of squares

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

what is partial eta squared?

A

proportion of variance that is uniquely explained by each variable -> finding out how strong / big the effect of your variables
- factorial ANOVAs used (tells you in estimates of effect size ANOVA table)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

why is partial eta squared used for factorial ANOVAs?

A

more than one variance and effect size

24
Q

what does p value measure?

A

the significant percentage of variance that is explained

25
what is the equal for partial eta squared?
Partial eta2 = SSeffect / SSeffect + SSerror(/residual)
26
what is partial eta squared??
ratio of variance associated with an effect, plus that effect and its associated error variance
27
what are (partial) eta squared scaled between?
0 (none of the variance) and 1 (100% of the variance)
28
how do you report effect size?
alongside other statistical information: F(df)=…., p=…., n2 =…
29
can you have a significant effect, yet a really small eta squared?
yes -> in this case, think about how meaningful this is and what else could be having an effect on your dependent variance (if there is only a small significant effect)
30
what is power analysis?
used to determine whether your design is appropriate and may find the effect you're looking for - can be confident about the result you are looking for -> by how powerful your design is
31
when is a power analysis run?
usually before you start collecting data
32
what does the 'power' of a statistical test do?
* ability to detect an effect when it is actually there (powerful enlightenment to be able to see an effect) * ability of a test to correctly reject the null hypothesis (can confidently say when zooming in there is nothing there)
33
what does power dependent upon / influenced on?
sample size, criteria for significance and effect size
34
sample size
the more participants = increased power and chance of finding a significant effect (if there is one) * useful because if we know how many subjects we need to detect an effect size at a given power
35
what is usually considered a good level of power?
around 0.8 - can help you understand that you may need like at least 17 subjects to achieve a significant effect at a power level of 0.8 [minimum number of participants to test in order to find an effect)
36
(power depending on) effect size
find an effect we know is there in reality * smaller effect size need more participants to achieve a higher power -> interaction between number of participants, power and effect size [if we know at least two of these, then we can estimate the other that we are trying to find]
37
(power depending on) power of significance [alpha level, a]
as you increase/decrease alpha level, it will influence how powerful your design is
38
what do we want to do about power?
make sure we have enough power to detect an effect ( be confident in that result, or be confident that their isn't a result -> reject null hypothesis it indeed shows a null result)
39
what are the dangers of an underpowered study?
* unpowered (too few participants) - lack of power to detect effect -> type 2 error (false negative) - increased chance of type 1 error (false positive) -> can't be confident if the same size is smaller etc. * If we estimate power across many published studies it is often worryingly low - e.g. Button et al (2013) estimated the median power of studies is around 0.08-0.31 * Low power explains failure to replicate - current replication crisis in Psychology (and other disciplines) -> and therefore we don't know who's right
40
power analysis also carries ethical issues. what ethical issues can this carry?
* Expensive * Inconvenient * Boring * Uncomfortable * Painful * Dangerous * Even Fatal (animal research)
41
what is a requirement for ethical approval now?
demonstrating your study has sufficient power * no justification for running a study if it doesn't stand a reasonable chance of being informative esp if painful, dangerous etc - wasting people's time is unethical too including causing someone discomfort if you are not confident in your result
42
what should you look into, in order to be relatively confidence about your findings?
pilot studies
43
what are four ways in which we can estimate effect size?
1. Guess 2. Pilot Study 3. Find Previous Research 4. Find or Conduct a Meta-analysis
44
1. Guess
* good if there's much literature on your phenomenon * you can guess / estimate the effect size of your experiment by using cohen's heuristic BUT it's not terribly satisfying nor super informative :/ [garbage in, garbage out model -> based on nothing] -> but can be used to say that even if the effect is tiny, we had enough participants to find an effect
45
2. Pilot Study
* Fewer participants to estimate the effect size * does not matter if study comes out significant -> still use it to get an estimate of r * you can use this estimate to project how many participants you should test * BUT estimates of effect sizes are largely unreliable with such small samples :/
46
3. Find previous research
* literature search -> studies investigating similar phenomenons * use their results to work out an expected effect size for your experiment * these studies will not use the exact same size :/ BUT a-priori power estimate are not exact - You can't predict the future! - Better than guessing!
47
4. Find or conduct a meta-analysis
* literature search * if there are many previous studies you can calculate an average effect size across all of their results * common in drug trials - often a meta-analysis will have already been published that you can take effect sizes from or work them out yourself from them
48
what is a problem with power analysis?
GIGO: Garbage in -> Garbage Out technique * If you make up the numbers you enter -> what you get out is not meaningful * estimates are not exact because you can't predict the future * for really complex designs (i.e. factorial), it is unlikely you will have sufficient precision in your estimates of effect size - main effect & interactions - power analysis may not be meaningful -> gets lower and lower when you get through really complicated designs -> should be interpreted with caution
49
what are the three types of power analysis?
* A priori * Sensitivity * Post Hoc
50
A priori
Calculate how many participants are required for a study (-> given effect size, power, alpha)
51
Sensitivity
Calculate minimum required effect size detectable (-> given power, sample size and alpha)
52
Post-Hoc
Calculate observed power (-> given effect size, alpha and sample size)
53
How to conduct a power analysis using G*Power
* Test Family: F Tests (everything ANOVA giving you an F ratio) [obvs T instead for T tests] * Statistical Tests -> different options for the ones we are looking at -> we will typically look at ANOVA (select the one which is most appropriate) * Select the Type of Power Analysis (Post Hoc, A Priori or Sensitivity) * Enter the study design in the input parameters section * find the output -> i.e. if only 0.3, not significant really
54
what will the critical F value tell us?
the minimum F ratio required to reach significance
55
Powerful analysis is useful in which situations?
* estimating how many participants you should recruit * determining the power of a study design * working out the minimum effect size a study could detect reliably
56
*may be stuff on cheat sheet online which is helpful

research methods 2 (8 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2025 Bold Learning Solutions. Terms and Conditions