Ilan's lectures

Question 1

Jacob Cohen 1962

Answer 1

First power analysis, Cohens d to measure power of study to find null.
Did a meta analysis and found research was much less likely to find small effect and less likely to find medium effects and most likely to find large effects, most papers with small but sig results dont publish, UNCOVERED THE PUBLICATION BIAS

Question 2

Stapel 2011 secret life of emotions

Answer 2

Tilburg uni dean well respected published in big journals,
famous study the secret life of emotions;
primed disgust or fear with superfast image display then saw if those emotions were meta cognitively realised and if they impacted word finishing tasks to choose disgusting/fearful words over neutral words

no actual data collection happened, the whole thing was faked

Question 3

Bem 2011 ESP

Answer 3

book of evidence for ESP future telling, find the erotic picture, small power effect found, however furthur analysis including Bayesian analysis and non directional t test found no effect (Wagenmakers 2011), other studies found faulty procedures

Question 4

P hacking

Answer 4

a body of studies should have its own distribution of p values, over item all possible valid ps would be found, however meta analysis show that in fact there is a very uneven distribution of p’s implying p hacking

Question 5

R index

Answer 5

how replicable the study is

Question 6

Researchers degree of freedom

Answer 6

ability to analyse using the best tool for the results, that will give you a positive result

Question 7

Data mining methods

Answer 7

change sample size
meausre many report on one DV chosen post hoc
change type 1 error rate post hoc

Question 8

Publication Bias

Answer 8

Journals want sexy (new, counterintuitive, sounds nice, easy to write about) results

Question 9

Problem with Meta analysis

Answer 9

they only use published data, which is subject to piblication bias, so meta analysis compound these issues

Question 10

solutions to problems with replicability and bias

Answer 10

pre registration
OPEN source, data, access, methodology, peer review, education resources
sample non -WEIRD’s
focus on boundary conditions not point analysis
deal with systemic issues in the industry ie pressure to publish and publication bias

Question 11

CUDDY 2010 power pose study

Answer 11

fail whale
small sample size
DV changed throughout
p hacked
outliers cut out
gender not properly dealt with (measuring hormones as DV)