2. Writing and Rating a Paper

Question 1

Explain the process of peer review

Answer 1

1. Authors submit a paper
2. Journal editor can reject straight away
3. Paper is allocated to reviewers (single-blind)
4. Critical evaluation; coherence, references, validity, relevance, suitable methodology, adequate sample, ethics, appropriate analysis, limitations, justified conclusions
5. Judgement; paper quality, novelty vs replication, suitability for journal
6. Reccomandations are sent
7. Editors make final decision
8. Accepted and published or not (can send to other journals)

Question 2

What are the criticisms of peer review?

Answer 2

1. Based on trust and ethical behaviour (plagiarism, faking data/results, fabricated peer reviewers)
2. Publishers may only work with a small pool of trusted reviewers (slows process, may not be experts in the field, reviewed under pressure)
3. System can be resistant to publishing novel/challenging research
4. Scientific culture (pressure to publish frequently can lead to low quality, pressure to publish in high quality journals leads to reduced dissemination)

Question 3

What is publication bias?

Answer 3

1. Significant results prioritized for publication (null results are also important)
2. Representativeness of published results
3. Positive outcome bias

Authors may without negative findings or hypothesis testing, amend analysis plan or manipulate findings

Question 4

Why is replication important?

Answer 4

Good evidence that a concept is generalizable, open science collaboration created a reproducibility project in 2011.

Question 5

What is open peer review?

Answer 5

Reviewers are not anonymous to increase transparency and promote a greater effort for reviewers to be unbiased and thorough. Also gives credit to reviewer’s work.

But can mean reviewers are overly cautious, less dispassionate, and can frustrate collaborative attitudes.