Week 2 - Effect Sizes & Power Calculations

Question 1

Study design where only a single person is described

Answer 1

Case study

Question 2

Study design where multiple people are described

Answer 2

Case series

Question 3

Type of study that might be used in clinical psychology or neuropsychology (e.g., rare disease or particular type of acquired brain injury etc.).

Answer 3

Case study or case series

Question 4

Study design involving analysis of non-numerical data obtained through interviews, focus groups, observation, and often aims to identify themes related to a person’s experiences

Answer 4

Qualitative study

Question 5

Study design that is descriptive (trying to describe experiences of a person/group rather than trying to draw inferences to a larger population)

Answer 5

Qualitative study

Question 6

Benefit of qualitative studies

Answer 6

Rich for hypothesis generation

Question 7

Disadvantage of qualitative studies

Answer 7

Can’t draw inferences to different population - can only draw conclusions about the particular people studied (that were actually interviewed or observed).

Question 8

Study design that includes case-control, cross-sectional, cohort (longitudinal) studies and where the researcher cannot control the independent variables (can’t randomly assign people to conditions) and they compare an outcome(s) between naturally occurring groups

Answer 8

Observational study

Question 9

In an observational study, why can you draw inference about relationships, but not cause and effect?

Answer 9

Because didn’t manipulate the independent variables and then control around that (lacks random assignment)

Question 10

Study design that typically uses survey methodologies (e.g., questionnaires)

Answer 10

Observational study

Question 11

Study design that includes controlled and field experiments, participants are randomly allocated to conditions and typically involves a control or baseline condition with no manipulation

Answer 11

Experimental study

Question 12

Why do experimental studies permit inference about cause and effect

Answer 12

Because they have random assignment (and therefore are manipulating/intervening with the independent variables)

Question 13

Quasi-experiment

Answer 13

some variables within study cannot be randomly allocated between conditions, but other variables are e.g., gender -> if 1 of 2 variables in experiment and the other is randomly assigned (still falls under the experimental study design)

Question 14

Study design that involves:
1. literature review
2. Systematic review
3. Meta-analyses

Answer 14

Review study

Question 15

Study design that involves the study of research literature (not people) and provides an overview of the state of knowledge in a domain

Answer 15

Review study

Question 16

Type of review study that is a selective review (hence, potential for bias)

Answer 16

Literature review

Question 17

Type of review study which aims to canvas all knowledge on a topic using replicable scientific methods

Answer 17

Systematic reviews

Question 18

Type of review study which involves the statistical analysis of similar outcome measures from previous studies and summarises effect sizes in the literature

Answer 18

Meta-analysis

Question 19

Alpha definition

Answer 19

Probability that we’re willing to accept that we will falsely claim there is an effect when there wasn’t “willing to accept a 5% chance of making a type I error”

Question 20

What is the broad formula for power?

Answer 20

1 - beta

Question 21

Power

Answer 21

Chance of detecting an effect if one is there

Question 22

Probability of correctly rejecting the null hypothesis

Answer 22

Power

Question 23

Probability of rejecting the null hypothesis when the null hypothesis is false

Answer 23

Power

Question 24

Probability that a statistical test will detect an effect that is present

Answer 24

Power

Question 25

Probability of avoiding a type II error

Answer 25

Power

Question 26

Probability that the observed effect is due to chance

Answer 26

Alpha (Type 1 error rate willing to accept)

Question 27

Magnitude of the treatment effect relative to observation noise

Answer 27

Effect size

Question 28

Calculation that determines the minimum sample size needed to satisfy the desired alpha, power and effect sizes

Answer 28

A priori power calculation

Question 29

Calculation that determines the smallest effect size that can be detected for the given alpha, power and sample size

Answer 29

Sensitivity analysis

Question 30

When would you use an a priori power calculation

Answer 30

When want to figure out the minimum sample size needed to detect an effect if it is there (conducted before study)

Question 31

When would you use a sensitivity analysis

Answer 31

When there were/are constraints on the maximum number of people that could be recruited and want to know what the smallest effect size I can hope to detect is.

Question 32

Calculation that determines the alpha level that satisfies the power, effect size and sample size (not common practice)

Answer 32

Criterion analysis

Question 33

Calculation that determines the power that the test had to detect an effect if one was there

Answer 33

Post hoc power calculation

Question 34

When would you use a post hoc power calculation

Answer 34

After you’ve conducted the study and want to see what power the test had to detect an effect if it was there.

Question 35

Using the below sources will help you find what?
- Pilot data
- Previous study similar to yours
- Meta-analysis
- Conventions (e.g., Cohen’s d, small/ medium/ large)
- Other e.g., drawing inspiration from real life (given observations)

Answer 35

An effect size

Question 36

True / False - power is a property of a hypothesis or research question

Answer 36

False - it is a property of a statistical test

Question 37

Purpose of a priori power analysis

Answer 37

To define the minimum sample size for a particular statistical test

Question 38

Which has more power - parametric or non-parametric tests

Answer 38

Parametric

Question 39

Which has more power - single variable studies or those with multiple independent variables

Answer 39

Single variable studies -> those with more variables need more participants to achieve acceptable power

Question 40

Effect size

Answer 40

How big or strong the difference/relationship is between groups

Question 41

Power

Answer 41

Chance of finding an effect if one is really there

Question 42

Type I error

Answer 42

Saying there was an effect when there really wasn’t

Question 43

Type II error

Answer 43

Saying there wasn’t an effect when there was

Question 44

What happens to the sample size required to detect an effect if you chance from a two-tailed test to a one-tailed?

Answer 44

Sample size decreases for a one-tailed test

Question 45

Why do you need more participants for a two-tailed test than a one-tailed?

Answer 45

One tailed tests possess greater power. This is because concentrating alpha (0.05) at one end of the distribution, instead of splitting over the two ends to look for an effect in either direction.

Question 46

Do smaller or larger effect sizes require more participants

Answer 46

Smaller effect sizes require more participants to detect an effect

Question 47

Why do smaller effect sizes require more participants than larger effect sizes

Answer 47

Smaller effect sizes are harder to detect, so need more people
think - if someone is lost in a forest and calling for help; if the person needing help had a quiet voice, then it would take a lot more people in the rescue team to spread out and hear them calling for help as opposed to only a few people in the rescue team

Question 48

How does increasing the difference between the means of two groups affect sample size/effect size

Answer 48

Larger difference between group means = larger effect size. Larger effect size = less participants required

Question 49

Why does increasing the difference between the means for two groups require less participants?

Answer 49

Because Group A’s mean moves further away from Group B’s mean. which increases the effect size. In terms of a signal:noise ratio, having a greater difference between the means increases the signal relative to the noise (the two distributions have less overlap).

note: signal refers to effect you’re studying, while noise refers to variability or random fluctuations in the data that aren’t related to the effect that you’re studying

Question 50

How does decreasing the standard deviation in a group affect sample size?

Answer 50

Decreases the required sample size

Question 51

Why does decreasing the standard deviation for a group decrease the required sample size?

Answer 51

Decreasing standard deviation increases effect size, requiring less participants.
Decreasing the SD, lowers the amount of noise in the signal:noise ratio (measurements are more precise, distributions are pulled in), increasing effect size

Question 52

Is the sample size larger when trying to detect a main effect or an interaction effect?

Answer 52

Sample size is larger when trying to detect an interaction effect

Question 53

Why is sample size larger when trying to detect an interaction effect compared to the main effect?

Answer 53

Interaction effect is harder to find. When looking for an interaction effect, you’re looking for it against more noise (more groups being compared).

*Consider looking for a mean in a single group. Only the noise of that group affects your estimate of the mean. Now consider looking for a difference of means between 3 groups. The difference (signal) is subject to more noise, so we are less likely to reliably detect it. Now consider looking for a difference of differences of means, that is, a two-way interaction. That involves noise from 9 independent groups - the difference of differences is subject to yet more noise, so we are even less likely to reliably detect it [conversely, when looking at the main effect we’re collapsing across groups or levels of one IV and only looking at the difference between 3 groups so therefore less noise].