The Role of Peer Review in the Scientific Process Flashcards
What is Peer Review?
Peer Review is the assessment of scientific work by others who are experts in the same field.
The intention of peer reviewing is to ensure that any research conducted and published is of high quality. It is in the interest of all scientists that their work is held up for scrutiny and that any work that is flawed is detected.
The main Purpose of the Peer Review includes:
1) To allocate research funding; independent peer review takes place to decide whether or to to award funding for a proposed research project. (involve explanation as well otherwise may not award full marks).
2) To validate the quality and relevance of research; all elements of the research are assessed for quality and accuracy.
3) To suggest amendments and improvements; reviewers may suggest minor revisions to the work and thereby improve the report. In extreme circumstances, they may conclude that the work should be withdrawn.
The Process of Peer Review
1) Peer Review takes place after the research has been conducted but before its publication. It is usual for several expert reviewers to be sent copies of the researcher’s work by a journal editor.
2) Independent scrutiny of the work is conducted by other psychologists working in the same field.
3) The work is considered in terms of its validity, significance and originality.
4) Appropriateness and the methods and designs used is also assessed.
5) Although double-blind and open review also exist, the usual form of peer review is single-blind which involves the names of the reviewers not being revealed. The anonymity of the reviewer allows for an unbiased review free from interference by the researcher.
6) The reviewers report back to the editor, highlighting weaknesses or problem areas, as well as suggestions for improvement if necessary.
7) There are generally four options for reviewers to recommend: accept the work unconditionally, accept with some improvements, reject with suggestions of revision and resubmission and reject outright.
8) The journal editor makes the final decision whether to accept or reject the research report based on the reviewers’ comments and recommendations.
Strength
Essential part of Research
It is clear why peer reviews is essential - without it we don’t know what is mere opinion and speculation, and what real fact is. Peer review should ensure the quality of research so that flawed or fraudulent studies do not find their way into the public domain or get put into practise. The consequences of false or unscientific research being accepted as true can be serious, not least because many other scientists’ subsequent may be built upon the fact of the original research being accepted as true.
The Cyril Burt Affair - An example of scientific fraud
In the early 1950s, the eminent psychologist Sir Cyril Burt published results from studies of identical twins that was used as evidence to show that intelligence is inherited. Burt (1955) started with 21 pairs of twins, later increasing this to 42 pairs of twins reared apart. In a subsequent study, Burt (1966) increased his sample to 53 pairs of identical twins raised apart, reporting an identical correlation of (0.771) to the earlier twin study.
The suspicious consistency of these correlation coefficients led Kamin (1974) to accuse Burt of inventing data. When a reporter, Oliver Gillie (1976), tried but failed to find two of Burt’s research assistants this appeared to confirm the underlying fraud and
Burt was publicly discredited.
The Burt affair is particularly worrying because his research was used to shape social policy. Burt helped to establish the Eleven-Plus examination used in the UK to identify which children were brighter and should go to grammar school rather than secondary moderns. He argued that since IQ was largely genetic, it was appropriate to test and segregate children into schools appropriate to their abilities. The discovery of Burt’s fraud may have subsequently contributed to the move away from grammar schools.
Weakness
Issues with Anonymity
Anonymity is usually practised so that reviewers may be honest and objective. However, this is not always achievable, especially in highly specialised and recognisable research. On the other hand, anonymity may have the opposite effect if reviewers use the veil of anonymity to settle old scores or bury rival research. Reviewers have been accused of not accepting research ,so that their own studies can be published, and even of plagiarising research and passing it off as their own. As a result, some journals now favour ‘open reviewing’ where both author and reviewer know each other’s identity.
Weakness
Publication Bias
Journals tend to prefer to publish positive results (results there the research hypothesis has been accepted), possibly because editors want research that has important implications in order to increase the standing of their journal. In addition, there is evidence of ‘institution bias (tendency to favour research from prestigious institutions) and gender bias (the tendency to favour male researchers). This results in a bias in published research that in turn leads to a misperception of the true facts.
Weakness
Preserving the ‘Status Quo’
To maintain the status quo is to keep the things the way they presently are. Peer review results in a preference for preserving the ‘status quo’, research that goes with existing theory rather than dissenting or unconventional work. This fits with Kuhn’s idea that scientists share a set of ideas and assumptions about their discipline (paradigm) and tend to agree with findings that support this paradigm and ignore those that contradict it. This is a problem because considering contradictive evidence and questioning existing paradigms is how science progresses.
