Week 4 Flashcards
1
Q
Early philosophers
A
- First big ideas about mind and science
2
Q
Enlightenment
A
- Growing questions about mind, mechanism, empiricism
3
Q
Early psychologists
A
- How to be experimentalists
- Study perception, consciousness, intelligence
4
Q
Psychoanalysts
A
- The importance of the unconscious, inner conflict
5
Q
Behaviourists
A
- No more mind silliness
- Behaviour only
6
Q
Cognitive revolution
A
- The mind is back in psychology
- Study as information processing
7
Q
Paradigm shift
A
- Dominant schools of thought about how to study the mind scientifically have changed
- Zeitgeists
- Often periods of upheaval, revolution
8
Q
Reproducibility
A
- The extent to which consistent results are observed when scientific studies are repeated
- Major demarcation between science and pseudo-science
- Scientific claims should not gain credence by virtue of status/authority of their originator
9
Q
Science
A
- Systematic observation
- Ruthless peer review
- Considers all evidence
- Invites criticism
- Repeated results
- Limited claims
- Specific terms, operational definitions
- Engages community
- Changes with new evidence
- Follows evidence where it leads
10
Q
Pseudoscience
A
- Anecdotal evidence
- No peer review
- Considers only positive evidence
- Dismisses criticism
- Non-repeatable results
- Grandiose claims
- Vague terms and ideas – science-y jargon
- Isolated
- Dogmatic and unyielding
- Starts with a conclusion, works back to confirm
11
Q
How to collect data
A
- Generate a hypothesis
- Is it interesting
- Collect some data
- Maybe the first study doesn’t work so you fix it by changing some variables
- Repeat step 3 until you have enough studies to publish
12
Q
Stapel – 2010
A
- Prolific Dutch Social psychologist was investigated for fraud
- He often supplied the data to his grad students
- His grad students working in the lab remarked that stats for different studies showed similar means and SDs
- After investigation, admitting his fraud and found 25 published papers were based on fabricated data
- 58 papers were retracted
13
Q
Daryll Bem
A
- ESP study
- Claimed people had precognition
- Picking between pictures behind curtains
- The study had issues of reproduction
14
Q
Common bad research practices
A
- Stopping data collection once p is smaller than .05
- Analyse many measures but only report significant ones
- Collect and analyse many conditions but only report significant
- Using covariates to get significance
- Excluding participants
- Transforming data to get p<.05
15
Q
Open science collaboration
A
- 100 replications of pieces of research from 3 prominent journals
- Formed in 2011 with around 60 members
- Grew to 270 scientists from over 50 countries
- 97% of original studies reported significant effects
- 36% of replications had significant effects in the same direction
16
Q
Replication crisis factors – how did the crisis happen
A
- Enormous pressure and incentive to produce many papers
- Over-interest in counter-intuitive findings as sexy
- Confirmation biases by researchers promoting questionable research practices
- Lack of accountability and transparency
17
Q
Confirmation bias
A
- Tendency to seek out information that verifies your theory (validation)
- And not seek out information that falsifies your theory (falsification)
18
Q
Drawbacks of the replication crisis
A
- Failed replications may be the new trend
- It creates a culture of paranoia and moral righteousness
19
Q
Dan Gilbert – reaction to the crisis
A
- So-called replicators are shameless little bullies and second stringers
20
Q
Power pose replication crisis
A
- Carney, Cuddy and Yap 2010
- As evidence has come in over these past 2+ years
- My views have updated to reflect the evidence
- I do not believe that power pose effects are real
21
Q
Schnall replication crisis
A
- One of her major papers failed to be replicated
- Schnall wrote a response commenting on damage to her career
- Roberts commented that damage to her career was less important than the PhDs she ruined for being honest
22
Q
Increase replication
A
- We must ensure that findings are robust and replicable
- Direct/conceptual replications should be a part of research pipeline
23
Q
Beware P-hacking
A
- Exploring researcher degrees of freedom to find a significant effect on
- Changing degrees of freedom to find a significant effect
- Implicit bias or explicit data manipulation
24
Q
Definitions of power
A
- The probability of finding an effect in your study if the effect is real
- The probability that a test of significance will detect a possible deviation from the null hypothesis, should such a deviation exist
- The probability of avoiding a type 2 error
25
Q
Boost your power
A
- Studies can be underpowered
- Due to misunderstanding of power
- Large studies are more expensive and time-consuming
- We need to publish more papers, more frequently
26
Q
How do you increase power
A
- Larger sample sizes
27
Q
Open data principles
A
- Making data, materials and analysis available online
- So that others can replicate, check and reproduce your work
28
Q
Confirmatory/exploratory research
A
- Confirmatory – hypothesis testing
- Exploratory – studying around the topic
- Exploratory research is okay
- But must not be presented as confirmatory
- Should be followed by confirmatory
29
Q
HARK-ing
A
- Hypothesising after results are known
30
Q
How to conduct confirmatory research
A
- Decide study details a priori
- Hypotheses to test, number of subjects, conditions, DVs
- Only then start recruiting
- Pre-register your study
31
Q
Open science practices in teaching
A
- Ensure the next generation moves on from the reproducibility crisis
- Teach the importance of conducting well-powered studies
- Encourage critical evaluation of published studies in terms of open science practices
- Open science practice leads to more reliable, reproducible science
32
Q
Open science practices as reviewers
A
- Signatories will not offer comprehensive review for any manuscript that does not meet the minimum requirements
- Neither will they recommend the publication
- Stimuli and materials should be made publicly available
- Data should be made publicly available
- Documents containing details for interpreting any data files or analysis code should be made available
- The location of all these files should be advertised in the manuscript and all files should be hosted by a reliable third party
33
Q
Incentives for pursuit of new ideas
A
- Publications, grant income, employment, promotion, tenure, fame
34
Q
Issues with rewarding science practices
A
- Poor motivators
- May prompt bad practices for fame
35
Q
Goals for reward open science practices
A
- Be tolerant of lower output if doing correctly
- Reward food practices for pre-registered studies regardless of outcome
- Reward good practices for high powered studies
- More time for research
36
Q
Solutions to the replicability crisis
A
- More replications
- Beware of p-hacking
- Boost your power
- Open data, open materials, open analysis
- Conduct pre-registered confirmatory studies
- Incorporate open science practices in teaches
- Insist on open science practice as reviewers
- Reward open science practices
