Scientific Process + Peer Review Flashcards
how is peer review linked to the scientific process?
the scientific process involved in psychology is when knowledge is developed through conducting research and sharing the findings of this research with other scientists and the public
peer review is a central part of this process whereby the scientific quality of findings are judged prior to publication
it is in the interest of all scientists that their work is held up for scrutiny and any work that is flawed or fraudulent is detected and the results of such research is ignored
what is peer review?
the practice of using independent experts to assess the quality and validity of scientific research and academic reports
it is essentially the assessment of scientific work by others who are experts in the same field
the intention of peer reviewing is to ensure that any research conducted and published is of high quality
usually there are a number of reviewers for each application or assessment whose task is to report on the quality of the research and then their views are considered by a peer review panel
three main purposes of peer review
the Parliamentary Office of Science and Technology (2002) suggests that peer review serves three main purposes….
- allocation of research funding
- publication of research in academic journals and books
- assessing the research rating of university departments
allocation of research funding
research is paid for by various government and charitable bodies
the overall budget for science research in the year 2015/16 was set at £5.8 billion
the organisations spending this money have a duty to spend it responsibly, therefore public bodies such as the Medical Research Council require reviews to enable them to decide which research is likely to be worthwhile funding
publication of research in academic journals and books
scientific and academic journals provide scientists with the opportunity to share the results of their research
the peer review process has been used for this propose since the middle of the 20th century as a means of preventing incorrect or faulty data entering the public domain
prior to the idea of peer-review, research was simply published but now any research published on an academic level must be shown to be valid and of high quality
assessing the research rating of university departments
all university science departments are expected to conduct research
this research is assessed in terms of quality using the Research Excellence Framework
future funding for the department depends on receiving good ratings from the REF peer review
peer review and the internet
the sheer volume of information available on the internet means that new solutions are needed in order to maintain the quality of information
scientific information is now available in numerous online blogs, online journals and wikipedia
such sources are largely policed by the wisdom of crowds approach whereby readers decide whether the information is valid or not and post comments or edit entries accordingly
several online journals such as Philica ask readers to rate articles and papers are ranked on the basis of peer reviews which can be read by anyone
the term ‘peer’ is coming to mean ‘everyone’ on the interest, this is a more egalitarian system where everyone can check for validity but possibly at the cost of quality
fraud in psychology
CYRIL BURT
in the 1950s, British psychologist called Cyril Burt published results from studies of twins that was used to show that intelligence is inherited
he gradually increased the sample used from 21 pairs of twins raised apart to to 53 pairs and reported an identical correlation to his earlier twin study
the suspicious consistency of this correlation led many to accuse him of inventing data
a reporter tried and failed to find two of Burt’s research assistants who didn’t actually exist and this appeared to confirm the underlying fraud and Burt was publicly discredited
this case is particularly worrying because his research was used to shape social policy — for instance, he helped to establish the 11+ examination used in the UK as he argued that since IQ was largely inherited it was appropriate to test and segregate children into schools suitable for their abilities
demonstrates the importance of peer review
fraud in psychology
LESLIE JOHN
in 2012, Leslie John and colleagues surveyed 2000 psychologists and asked them to anonymously report their involvement in questionable research practices
they found that 70% said they cut corners in reporting data and 1% admitted to falsifying data
problems with fraud in psychology
in the future, people may be less likely to trust scientific data
data from fraudulent studies remains published, the journals involved usually publish retractions and state that the evidence is flawed but there are still people who continue to use the faulty data without knowing that it has been discredited
evaluation of peer review
the importance of peer review
finding an expert
anonymity
publication bias
preserving the status quo
cannot deal with already published data
importance of peer review
in the allocation of research funding, publication of research in academic journals and books and assessment of the research rating of university departments
without it, people would not know what is mere opinion and speculation and what is rigourously researched data
it provides a means of establishing the validity of scientific research and helps people know which data to trust and which to dismiss
finding an expert
certain features of the peer review process can be criticised
for instance, it is not always possible to find an appropriate expert to review a research proposal or report
meaning that poor research may be passed because the reviewer did not truly understand it
anonymity
anonymity is usually practiced so that reviewers can be honest and objective
however it may have the opposite effect, reviewers may use the veil of anonymity to bury rival research
research is conducted in a social world where people compete for research grants and jobs, so social relationships inevitably affect objectivity
so much so that some journals now favour open reviewing were both the author and the reviewer know each other’s identity
publication bias
Richard Smith, previous editor of the British Medical Journal, commented that “peer review is … highly subjective, prone to bias, easily abused … and almost useless at detecting fraud”
journals tend to prefer to publish positive results, possibly because editors want research that has important implications to feature in their journals in order to increase the standing of the journal
this results in a bias in published research that leads to a misperception of the true facts as many negative results tend not to be published
furthermore, it appears the journals avoid publishing replications of previous studies which is a fundamental part of research validation
Ritchie et al (2012) submitted a replication of a study on paranormal phenomena and found that it was not even considered for peer-review
this seems to suggest the journals are much like newspapers in seeking eye-catching stories
