Week 4 - The Publication Process & Multiverse Flashcards
What is the publication process
Reserachers explain their methods and findings in a paper.
* Pre-prints server
* Conference
* Scientific journal
What are pre-prints?
- Public servers
- No organized feedback or eval
- Often researchers will also aim to public the work as conference proceeding or journal article
- Open access to papers
What are journal articles
Contrary to pre-prints often no open access, especially for journal articles.
* Paid subscription
* Some journals are pay-to-publish
* Some journals do both (double-dipping)
What are Conference proceedings
- Quality of outlets can vary considerably
- Different indicators have been proposed. “Impact factor”
- Be wary of predatory journals
How to set up a conference proceeding
- Identify suitable vanue
- Editor/meta-reviewer assess fit, relevance
- Peer-review
- Editorial decision
- Authors revise their papers
What is desk reject?
Researchers need to look for different vanue/outlset or not publish the paper.
What are the 3 types of peer-review
- Single-blind
- Double-blind
- No blinding
What is single blind peer review
Reviewer remain anonymous
What is double-blind
both reviewer and auathor remain anonymous
what is no blinding
identities of both reviewers and authors are known to each other
What is scooping
The circumstance in which research is published by a researcher or researchers before a rival team can publish theirs on the same topic
What could be the an editorial decision
- accept as is
- minor revision
- major revision
- reject
Decisions by editors and reviewers can be driven by the outcome of the study. Add 2 more things
- Significant effects or “positive results” are more likely to be accepted
- Leads to publication bias and the file-drawer effect
What is the file-drawer effect
This term suggests that results not supporting the hypotheses of researchers often go no further than the researchers’ file drawers, leading to a bias in published research.
Why do researchers engage in QRPs?
- Null effects are not deemed interesting or publishable
- Important to publish and get positive results
- Need funding, contracts, promotion
Why would someone commit a QRP without their knowledge
- They didnt know
- Cognitive bias (Hindsight bias/ Confirmation bias)
What are the consequences of QRPs
- Replicability issues
- Overestimation performance
- Distrust in science
How to avoid QRP?
5 points
- Better training
- 21-word solution
- Blind analysis
- Preregistration
- Multiverse or sensitivity analysis
What is preregistration
- description of the hypothesis you want to test, the study’s design and analysis plan
- Sufficiently specific and precise
- BEFORE data collection or data analysis
- Public, either immediately or down the road
How does preregistration help?
- Need to think ahead
- A shield against cognitive biases
What is a registered report
when a peer review is conducted prior to collecting and analysis data
What is multiverse analysis
systematically sampling a vast set of model specifications, known as a multiverse, to estimate the uncertainty surrounding the validity of a scientific claim
