Fixed VS Random Effects

Question 1

FIXED EFFECTS ANALYSES

Answer 1

• used when you want to know if individual/precise conditions have an effect on performance
• ie. only interested in specific lvls of IV featured in dataset
• F-ratio calc takes error term from within those groups/conditions defined by said specific IV

Question 2

RANDOM EFFECTS ANALYSES

Answer 2

• used when values of experimental conditions have been sampled at random from wider pop of dif
• ie. we want to generalise observed effect to other possible lvls of IV
• F-ratio calc takes error term from whole sample (ie. all groups/conditions defined by all IVs in design)
• concern more w/effects of varying dimension under investigation than w/specific values tested in study

Question 3

WHEN TO USE RANDOM EFFECTS

Answer 3

• if only interested in if there is some effect of experimental manipulation (ie. delaying recall in memory task) then it’s reasonable to regard individual values of delay as of little importance in own right
• reasonable to consider that these delay values might’ve been sampled from population (wide range) of possible values (ie. 10/30/50s)
• if so, delay = random effect
• BUT…

Question 4

WHEN TO USE FIXED EFFECTS

Answer 4

• rare that values of variable = genuinely sampled at random as such
• aka. in ALMOST ALL circumstances = appropriate to treat variables as fixed effects
  ADVANTAGE
• results more likely to be statsig
  DISADVANTAGE
• sig findings cannot be generalised to dimension as whole
• ie. statsig recall delay cannot be generalised into a reliable effect

Question 5

RANDOM EFFECTS EVALUATION

Answer 5

• assumes relation between IV/DV = consistent throughout full range of possible values of IV (ie. ignored dif between linear/quadratic)
• BUT… relation may only be consistent for part of range/may have 2+ distinct relations consistent w/particular ranges
• aka. may not be appropriate to generalise effect to entire dimension BUT ofc cannot say this w/o relevant research

Question 6

EXAMPLES

Answer 6

GENDER
- fixed (ie. male VS female VS non-binary)
TREATMENT
- fixed (ie. CBT VS mindfulness VS no therapy)
PRIMED CONDITION
- fixed (aggressive VS friendly VS no prime)
LOCATION
- both (specific locations = fixed; regional = random)
AGE
- both (specific age = fixed; keystage/development = random)

Question 7

RANDOM EFFECT (EXAMPLE)

Answer 7

• typical experiment = psychologist tests 10/20/30 dif pps
• rarely interested in knowing if specific people show specific data-patterns
• rather dif pps = normally sampled as randomly as possible to estimate variability within pop
• aka. pps = ALMOST ALWAYS random effect
• SPSS does it automatically in repeated measures (aka. why you have to type in data for each individual pp on separate row & not include pp number as sperate variable)

Question 8

STATISTICAL POWER

Answer 8

• chance that it will allow you to declare existence of main effect/interaction that is GENUINELY in the data
• aka. measure of test’s SENSITIVITY in detecting genuine effect
• expressed in probability value terms:
  POWER = 1 -> implies 100% chance test will detect effect (it it exists); 80% benchmark in psych
  POWER = 0 -> absolutely no chance of detecting effect

Question 9

EFFECT SIZE

Answer 9

• reflects strength of influence of IV on DV
• larger effect size = ^ difs in DV between groups defined by IV
• larger effects = more visible/easier to detect

Question 10

EFFECT SIZE VS SIGNIFICANCE

Answer 10

• effect size = magnitude of observed effect; significance only tells us an effect exists
• significance w/o effect size measures = like saying Empire State is taller than Eiffel but w/o stating how much taller
• recent call to abandon significance altogether & replace it w/effect size

Question 11

EFFECT SIZE x POWER

Answer 11

• more powerful analyses allow to detect smaller effect sizes
• larger effect sizes only need relatively low power analysis for detection
• small effect sizes need more powerful analyses for detection

Question 12

POWER INCREASES W/SAMPLE SIZE

Answer 12

• more powerful analyses allow smaller effect size detection
• more data points (ie. cases/pps) in each group -> more powerful analyses
• BUT… if there’s nothing there, it still won’t be found