replication crisis

Question 1

what is the replication crisis

Answer 1

methodological crisis where researchers have found that a significant proportion of scientific studies are difficult or impossible to replicate

Question 2

what is the impact of the replication crisis?

Answer 2

• led to questions about the creditability of research, affecting public trust in science
• decisions on non replicable research could have significant implications in areas like health, education, eduction and public policy
• inability to build on previous research can hamper scientific progress

Question 3

replication vs reproducibility

Answer 3

consistent results when a study is repeated vs the ability to achieve the same results when reanalysing the same data set

Question 4

what did Ioannidis’ research paper state?

Answer 4

a large number of published medical research papers contain results that cannot be replicated

responses to the paper suggest lower false positive rates

Question 5

what is HARKing?

Answer 5

hypothesising after the results are known

Question 6

what is publication bias

Answer 6

studies with statistically significant findings are more likely to be published

Question 7

what are voodoo correlations

Answer 7

unrealistically high correlations reported, particularly in brain imaging research

Question 8

what is P-Hacking?

Answer 8

manipulating data analysis to achieve statistical significance

Question 9

what did Diederik Stapel do?

Answer 9

found to have fabricated data in numerous studies

case brought significant attention to the potential for outright fraud in psychological research

Question 10

why was Bem’s ESP study problematic

Answer 10

published controversial study claiming evidence for precognition

could not be replicated

Question 11

what was the power pose study, and what was the issue with it?

Answer 11

participants in high power pose supposedly made “riskier bets” and had “higher testosterone” and “lower cortisol” compared to low power pose

however, this has not been replicated by several researchers

Question 12

what was the social priming study, and what was the issue?

Answer 12

elderly walking study cited over 300 times:
- involved PPs unscrambling jumbled lists of words to form coherent sentences
- when lists contained words pertaining to ageing and the elderly, PPs walked out of the lab more slowly

however, not been replicated

Question 13

what are some reasons for lack of replication

Answer 13

• small sample sizes: lower statistical power, increased chance of type 1 errors
• misuse of statistical significance: overestimating p-values
• P-Hacking: manipulating data analysis to achieve statistical significance
• publication bias
• selective reporting
• lack of data sharing

Question 14

what was the GSK and Paxil scandal

Answer 14

SSRI

misrepresented data on Paxil’s safety

criticised for ghostwriting and downplaying risks

Question 15

what has been the response to the replication crisis

Answer 15

promotion of replication studies

open science practices (advocating open access to data and methodologies)

preregistration of studies (study registered before collecting data to prevent selective reporting)