Issues in Psychology

Question 1

Define replication crisis

Answer 1

Highly noted studies in psychology are not replicable (raises the issue of methodology)

Question 2

Where else has the rep crisis been observed? and how has it been found (2)

Answer 2

Physics: Cold nuclear fusion
Medicine: only 4% of Genetic literature has been replicated
Ozone layer detection: Thought the instruments were faulty when examining the Antarctic ozone layer hole

1. funnel plots
2. systematic reviews

Question 3

Types of replication

Answer 3

Exact replication: Testing the study using exactly the same methods

Conceptual replication: Testing the theory of the study using different population, variables etc

Question 4

What did the Open Science Forum Collaboration (2015) find

Answer 4

only 36% of journals and articles have been replicated
Social psychology is the worst - 23%
Cognitive articles were the best (still bad) - 48%

Question 5

What area is under scrutiny particulary?

Answer 5

Behaviourist social psych
Using priming or cues to induce a subconcious response (e.g., priming intelligence to see if intelligence can be created)
E.g. primed professor or football hooligan - found professors did better on tests. This has not replicated (Shanks et al., 2013)

Question 6

Why might the replication crisis have occured? (5)

Answer 6

(Sheilds, 2000)

1. Psychology has become too preoccupied with ‘attention grabbing findings’
2. The media has driven a thirst for controversial publications
3. Perhaps the researchers performing the replication studies lack expertise and methodolgical skill
4. The original studies may have had smaller sample sizes (false positive is more likely to have occured). Or the sample was (very unlikely) but out of chance and pure luck, unrepresentative.
5. Junior researchers publish null findings because they need to become ‘known’, but senior academics may ignore null findings because they are uninteresting.
6. The results may have changed over time! E.g., in the 50’s attitudes towards female army officers would be different to today.

Question 7

Solutions to rep crisis (and example)

Answer 7

1. Encourage academics to followup on their studies (often proffessors etc are encourage to persue new and interesting findinds, and replication is considered a waste of time and resources as it does not reflect original thinking
2. Highlight and publicize replication studies more
3. Educate consumers of science to read more cautiously and with more scepticism
4. Revise publication companies: Some only publish short , single study and punchy articles. Textbooks too seek sexier studies
   Cancer Epidemiology decided to publish all findings, including null. Plus, condense findings into single page reports so they are consumer friendly.

Question 8

What is p hacking?

Answer 8

Dredging data to find an effect, when one may not actually exist

Question 9

What is publication bias? 3 facts about it

Answer 9

Tendency to publish only significant findings

1. Insignificant findings are two times less likely to be published than significant results (Sheilds, 2000)
2. Significant results are published faster
3. Studies with smaller sample sizes are more likely to be publish and last longer as ‘relevant’

Question 10

What other kinds of bias?

Answer 10

Reference bias: where researchers cite only significant literature to bolster the validity of their claims

Question 11

Why is the publication so bad!?

Answer 11

1. Can affect health care and treatment (medical literature uses a scope of literature to inform the field) not just a single study, so if the published research only represents half of the story….
2. Science will suffer, progress will be delayed

Question 12

What is okay to be excluded?

Answer 12

Studies with poor methodology, insufficient power, poor construct validity

Question 13

Cleophas (1999) (2)

Answer 13

1. Publishing everything may be bad because it may devaluate the current literature.
2. May be a waste of time - it may never be read anyway

Question 14

World Health Organization

Answer 14

1/10 people suffer from some form of mental illness worldwide

Question 15

Why do researchers shy away from using mental health service users?

Answer 15

1. Hard to get ethical approval
2. Underestimate their cognitive abilities and the validity of their contribution
3. May have to use gatekeepers - long process/more resources

Question 16

What is a gatekeeper and what are their responsibilities?

Answer 16

1. Act as a middle-man or liasion between participant and researcher
2. Maintain the integrity of the research methodology
3. Prevent extra or intense demands being placed on the participants
4. Make sure sufficient support, understanding and aims are provided, and that the researcher demonstrates an ability to understand potential problems
   (Keogh & Daly, 2009)

Question 17

Research that highlights need to revise mental health services?

Answer 17

Rosenhan (1973)

Question 18

Theory behind research that is still applicable today?

Answer 18

Labelling and differential treatment (echoes Money & Erhhart, 1979).
Such that once a person is labelled one thing, all of their behaviour is attributed to be because of that label. Mental health service uses may face stigma and exclusion from research because of this label, but their contribution may barely differ, if all, from the contributio of ‘normal’ patients.

Question 19

When is it important to check participants mental health though?

Answer 19

1. If their condition (e.g, dementia) is going to deteriorate over the course of the study.
2. If their condition is going to affect their cognitive ability, if they are unable to understand the research aim etc. Can establish with Mini Mental State Examination or deliberating with Alzheimers soc.

Question 20

Why important to use mental health service users?

Answer 20

To improve the system and treatment, their voices must be heard.

Question 21

Case study examples of exploitation (2)

Answer 21

1. Genie - overtaxed and overstudied. Although the damage done to her was abhorrent, she made linguistic, physical and cognitive progress. This was reversed by psychologists continual removal and placement of Genie. Her current whereabouts is unknown.
2. Money and David Reimer (John/Joan case). Saw an opportunity for perfect methodological testing (twin control, no mental health history, nature v nurture). He received hormone treatment and gender reassignment. Was ostracized by his peer (‘cave woman’), depressed and eventually committed suicide. His public report exposé slowed gender reassignment, but at what cost?
3. Loftus planted distressing false memories similar to other studies in Tennesse and Canada: e.g., drowned as a child, attacked by an animal. But justified by saying that their momentary discomfort was outweighed by the important contribution these findings make to science.

Question 22

Klein et al (2014)

Answer 22

Used 6000 participants, a more ‘deep and narrow’ approach.
The studies had to be simple to administer and done online.
The studies were examining effects - and 77% were replicated

Question 23

Camerer (2018)

Answer 23

investigated articles published in Science and Nature, 21 systematically chosen studies using 5x larger sample sizes.
62% were replicated

Question 24

Reactions to rep crisis

Answer 24

1. Doubt it existed
2. Crisis is overblown
3. The replications were not exact replications
4. The researchers were not skilled
5. Concern for the future (e.g., red tape. The bar for what is considered unethical is now lower, there is growing concern that oppressive regulations will slow scientific progression and dampen innovation).

Question 25

The narrative

Answer 25

Perhaps the ‘crisis’ narrative is too severe, and does not actually reflect the severity of the situation.
Secondly, perhaps this is a natural cycle for methodology, merely growing pains.