the replication crisis

Question 1

what is the replication crisis

Answer 1

the failure to replicate published findings

Question 2

what is the timeline of the replication crsis

Answer 2

1) published ESP (extra sensory perception) article that said people could predict the future
- uproar when this came out, the way you present research can appear a certain way
- author of this was famous for writing about how to get article published, but did this in immoral ways

2) confessions to fabricating data
- 30 publications deleted

3) failure to replicate famous studies
- could not replicate basic social psych studies (elderly words priming slower walking)

4) fewer than 50% of studies replicated

Question 3

what is the crisis in the field surrounding replication crsis

Answer 3

are a lot of phenomenon we found not real?
-why some things do not replicate, but they do actually exist?
and why somethings do not exist but get published anyways

Question 4

what are the null effect possibilities

Answer 4

1) too small sample size
2) inadequate manipulation
3) wrong hypothesis

Question 5

what are meaningful failures to replicate

Answer 5

when the OG study results fail to replicate but they actually do mean something

Question 6

why dont studies replicate

Answer 6

1) direct v. conceptual replication

2) questionable research practice

Question 7

what are the two kinds of replications

Answer 7

direct and conceptual

Question 8

what are direct replications

Answer 8

seeks to validate a particular finding with same method (exact same operational definitions and procedure)

• lots of little things could affect the results of replications (ex. different computer monitors)
• what appears to be identical might not be

Question 9

what are conceptual replications

Answer 9

seeks to validate the underlying theory or phenomenon with different methods (also occurs during “extension” studies that identify moderators etc)

• replication with different methods is stronger evidence a phenomenon exists
• moderators: effect exists for some people in some conditions but not all

Question 10

explain the example with naturalistic bias

Answer 10

people prefer success in others with natural talent rather than those with work ethic

• could not replicate it in Arcadia (different measures-so went to original measures)
• original research read news articles about pro musician who either worked hard than had natural talent (it worked)

Question 11

how can a direct manipulation attempt deviate too much

Answer 11

1) inadequate manipulations

2) moderators

Question 12

explain inadequate manipulations example in facial feedback

Answer 12

facial feedback hypothesis
-mouth in smile = more likely to find funny but with video cameras on face

** this did not replicate, participants were video recorded which affected the results**

self awareness moderates facial feedback

Question 13

explain moderators

Answer 13

the official effect may have been real only for that sample

Question 14

explain Brown’s example of questionable research practice

Answer 14

Brown could not replicate own findings and she used direct methods, it was the same and not fishy
-concluded there was a moderator (study about politics) depended on the political climate at the time (everything occurs in a broader context)

Question 15

explain common and questionable research practices

Answer 15

• failing to report all DVs
• collecting data after preliminary checking results
• only reporting studies that “worked”
• failing to report all conditions ran
• claiming to have predicted an unexpected finding

Question 16

explain researcher degrees of freedom and an example

Answer 16

researchers used to do poor practices, it was the accepted field standard bc they had this freedom but it has changed

ex. participants computer freezes, do you scratch the data

Question 17

explain low incentives for conducting replications

Answer 17

• published studies shown once without any replication attempts
• science as a public dilemma (whats good for the individual might not always be good for the group)
• its good to get new ideas published for individuals (do not replicate others to improve their own brands)
• its good to replicate studies for the field as a whole

Question 18

what are some strategies for change in the replication crisis

Answer 18

1) reward replications
2) pre-registering studies
3) making data publicly available
4) large sample sizes (reduces type one error)

Question 19

explain rewarding replications

Answer 19

motivate people to do this
-but if we reward them too much “someone can build a career on incompetence”, get credit for repeating someone else’s work, if something does not replicate it could be because of poor manipulations (build a career on trashing others work)

Question 20

explain pre-registering studies

Answer 20

• make methods and analyses ready in advance to public

- forces people to be honest

Question 21

what are the costs of the changes in the replication crisis

Answer 21

• the push for huge sample size hurts researchers at small colleges, researchers without grants, young researchers (big colleges, established people and the wealthy benefit)
• exploratory work will suffer (brand new ideas will likely fail)
• unrealistic high standards