Week 6 - The Replication Crisis Flashcards
What is exact replication?
Also called direct replication, a scientific attempt to exactly copy the scientific methods used in an earlier study in an effort to determine whether the results are consistent. The same - or similar - results are an indication that the findings are accurate
What is conceptual replication?
A scientific attempt to copy the scientific hypothesis used in an earlier study in an effort to determine whether the results will generalise to different samples, times or situations. The same - or similar results are an indication that the findings are generalisable
What are the wider implications for poor replication rates in psychology?
Erodes the credibility of psychological science
can be dangerous
Wastes resources
What are some reasons why a study may be non-replicable?
Bad science
Questionable research practices
Publication issues
Diversity in study populations
How is a cultural shift improving the replication crisis?
Open science
Institutional change
Journal policies
How does open science increase replication?
Pre-registration (prevents cherry picking) publication of registered reports, open data and open resources
How are institutions changing to increase replication?
The reward structure in academia has served to discourage replication. Replications are typically discouraged because they do not represent original thinking, instead, academics are rewarded for flashy studies that are highly cited and given prominence in media reports
How can journal policies increase replication?
Some journals do not publish straight replications - although this is changing, open data required, larger participant number, effect size required, supplementary material for detailed methods
Why was Galak et al’s replication study of Bem 2011 criticised?
Statistics
No correction for multiple comparisons.
Significant effects were attributed to an excessive familywise error rate (i.e, type I errors) leading to false positive
What field of psychology has the lowest rates of replication?
Social psychology (23%)
What percentage of neuropsychology/cognition studies are replicated?
48%
What is reduced when a study is replicated?
Effect size
When will a true effect size emerge?
After many replications (and even then as an approximation)
Why do we need a lot of high quality replications?
As true effect sizes only emerge after may replications
What does it mean to do “bad science” when replicating a study?
The original results might have been falsified - fraud (rare but it happens - leads to disgrace and job loss)
Unskilled scientists and did not follow procedures closely enough
wrong statistics either in the original or replicated study
