Week 4 - The Publication Process & Multiverse

Question 1

What is the publication process

Answer 1

Reserachers explain their methods and findings in a paper.
* Pre-prints server
* Conference
* Scientific journal

Question 2

What are pre-prints?

Answer 2

• Public servers
• No organized feedback or eval
• Often researchers will also aim to public the work as conference proceeding or journal article
• Open access to papers

Question 3

What are journal articles

Answer 3

Contrary to pre-prints often no open access, especially for journal articles.
* Paid subscription
* Some journals are pay-to-publish
* Some journals do both (double-dipping)

Question 4

What are Conference proceedings

Answer 4

• Quality of outlets can vary considerably
• Different indicators have been proposed. “Impact factor”
• Be wary of predatory journals

Question 5

How to set up a conference proceeding

Answer 5

1. Identify suitable vanue
2. Editor/meta-reviewer assess fit, relevance
3. Peer-review
4. Editorial decision
5. Authors revise their papers

Question 6

What is desk reject?

Answer 6

Researchers need to look for different vanue/outlset or not publish the paper.

Question 7

What are the 3 types of peer-review

Answer 7

1. Single-blind
2. Double-blind
3. No blinding

Question 8

What is single blind peer review

Answer 8

Reviewer remain anonymous

Question 9

What is double-blind

Answer 9

both reviewer and auathor remain anonymous

Question 10

what is no blinding

Answer 10

identities of both reviewers and authors are known to each other

Question 11

What is scooping

Answer 11

The circumstance in which research is published by a researcher or researchers before a rival team can publish theirs on the same topic

Question 12

What could be the an editorial decision

Answer 12

• accept as is
• minor revision
• major revision
• reject

Question 13

Decisions by editors and reviewers can be driven by the outcome of the study. Add 2 more things

Answer 13

• Significant effects or “positive results” are more likely to be accepted
• Leads to publication bias and the file-drawer effect

Question 14

What is the file-drawer effect

Answer 14

This term suggests that results not supporting the hypotheses of researchers often go no further than the researchers’ file drawers, leading to a bias in published research.

Question 15

Why do researchers engage in QRPs?

Answer 15

• Null effects are not deemed interesting or publishable
• Important to publish and get positive results
• Need funding, contracts, promotion

Question 16

Why would someone commit a QRP without their knowledge

Answer 16

• They didnt know
• Cognitive bias (Hindsight bias/ Confirmation bias)

Question 17

What are the consequences of QRPs

Answer 17

• Replicability issues
• Overestimation performance
• Distrust in science

Question 18

How to avoid QRP?

5 points

Answer 18

1. Better training
2. 21-word solution
3. Blind analysis
4. Preregistration
5. Multiverse or sensitivity analysis

Question 19

What is preregistration

Answer 19

• description of the hypothesis you want to test, the study’s design and analysis plan
• Sufficiently specific and precise
• BEFORE data collection or data analysis
• Public, either immediately or down the road

Question 20

How does preregistration help?

Answer 20

• Need to think ahead
• A shield against cognitive biases

Question 21

What is a registered report

Answer 21

when a peer review is conducted prior to collecting and analysis data

Question 22

What is multiverse analysis

Answer 22

systematically sampling a vast set of model specifications, known as a multiverse, to estimate the uncertainty surrounding the validity of a scientific claim