L11 Reproducibility and Open Science

Question 1

Reproducibility crisis

Answer 1

The replication crisis (or replicability crisis or reproducibility crisis) is an ongoing (2018) methodological crisis in science in which scholars have found that the results of many scientific studies are difficult or impossible to replicate or reproduce on subsequent investigation, either by independent researchers or by the original researchers themselves

Question 2

What counts as successful replication?

Answer 2

• Statistically significant results were a match, p=0.05

- Asked experts whether they think the results -indicate the same thing

Question 3

Publication bias

Answer 3

Lots of papers reporting positive results, finding what they were looking for, especially in psychology

Question 4

Type 2 error

Answer 4

• false negatives
• uncommon to be reported
• Usually only type 1 error rate reported - false positive

Question 5

P-hacking

Answer 5

• Checking statistical significance of results before deciding whether to collect more data
• Rounding off p-value to meet statistical significance threshold, presenting 0.053 as p<0.05

Question 6

Cherry-picking

Answer 6

• Failure to report dependent variables/relationships that did not reach statistical significance or otherwise “work”
• Failure to report conditions/treatments that did not reach statistical significance or otherwise “work”

Question 7

HARKing

Answer 7

• hypothesising after the results are known

- Presenting a post hoc finding as though it had been hypothesized all along

Question 8

Open science

Answer 8

• Allowing public access to science
• Visibility, transparency
• Make public hypothesis and research plan before the actual experiment, this way you’re committed to it
• Preregister your research