Peer review Flashcards
The role of peer review - the scientific process
Science is a process which enables humankind to get closer to understanding how the world, and the people in it, function.
Many elements of this process have evolved over the centuries to ensure that we uncover facts that can be relied on to build bridges, treat disease, raise psychologically healthy children etc.
One part of this process is peer review.
The role of peer review - peer review
Peer review is the assessment of scientific work by others who are experts in the same field.
The intention of peer reviewing is to ensure that any research conducted and published is of high quality.
Usually there are a number of reviewers for each application / article / assessment.
Their task is to report on the quality of the research.
Their views are then considered by a peer review panel.
The Parliamentary Office of Science and Technology (2002)
Suggests that peer review serves three main purposes.
Allocation of research funding.
Publication of research in academic journals and books.
Assessing the research rating of university departments.
The Parliamentary Office of Science and Technology (2002) - allocation of research funding
Research is paid for by various government and charitable bodies.
The overall budget for science research in the year 2015-16 was set at £5.8 billion.
(According to the government website in 2014)
The organisations spending this money have a duty to spend this responsibly.
Therefore, public bodies such as the Medical Research Council require reviews to enable them to decide which research is likely to be worthwhile.
The Parliamentary Office of Science and Technology (2002) - publication of research in academic journals and books
Scientific or academic journals provide scientists with the opportunity to share the results of their research.
The peer review process has only been used in such journals since the middle of the 20th century as a means of preventing incorrect or faulty data entering the public domain.
Prior to the idea of peer review, research was simply published and it was assumed that the burden of proof lay with opponents of any new ideas.
The Parliamentary Office of Science and Technology (2002) - assessing the research rating of university departments
All university science departments are expected to conduct research.
This is assessed in terms of quality.
(Research Excellence Framework, REF)
Future funding for the department depends on receiving good ratings from the REF peer review.
The role of peer review - peer reviews and the internet
The sheer volume and pace of information available on the internet means that new solutions are needed in order to maintain the quality of information.
Scientific information is available in numerous online blogs and journals.
To a large extent, such sources of information are policed by the “wisdom of crowds” approach.
This is where readers decide whether it’s valid or not, and post comments and / or edit entries accordingly.
Several online journals ask readers to rate articles.
The role of peer review - peer reviews and the internet (Phillica)
Phillica - online journal.
It asks readers to rate articles.
The papers are ranked on the basis of peer reviews and the peer reviews can be read by anyone.
However, as it’s on the internet “peer” means “everyone”.
This possibly affects accuracy and / or quality.
Richard Smith (1999)
Previous editor of the British Medical Journal.
“Peer review is slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects and almost useless at detecting fraud.”
Evaluation - finding an expert
It isn’t always possible to find an appropriate expert to review a research proposal or report.
Evaluation - finding an expert (Smith (1999))
Poor research may be passed because the reviewer didn’t really understand it.
Evaluation - anonymity
Anonymity is usually practised so that reviewers can be honest and objective.
However, it may have the opposite effect if reviewers use the veil of anonymity to settle old scores or bury rival research.
Research is conducted in a social world where people compete for research grants and jobs, and make friends and enemies.
Social relationships inevitably affect objectivity.
Some journals now favour open reviewing.
This is where both the author and the reviewer know each other’s identity.
Evaluation - publication bias
Journals tend to prefer to publish positive results.
This is possibly because editors want research that has important implications in order to increase the standing of their journal.
This results in a bias in published research that leads to a misperception of true facts.
Furthermore, it appears that journals also avoid publishing straight replications of a study, a fundamental part of research validation.
Ritchie et al (2012)
Submitted a replication of a study on paranormal phenomena.
They found that that it wasn’t even considered for peer review.
They suggest that journals are as bad as newspapers for seeking eye-catching stories.
Evaluation - preserving the status quo
Peer review results in a preference for research that goes with existing theory rather than dissenting or unconventional work.
