L7 Flashcards
What is required in order for us to have more reproducible science?
Openness and sharing as many parts of these features of the research process as possible
How would an open culture help reproducibility?
If we change the incentives for getting published we can change the the incentive to the quality of the research rather than the finding of significant results.
What were Robert Mertons 4 norms for how science should be done?
Communalism
Universalism
Disinterestedness
Organised skepticism
What did Mertons communalism refer to?
Openness and transparency in research
Science should be shared
What did Mertons universalism refer to?
Science should be independent of sociopolitical status
What did Mertons disinterestedness refer to?
Science should be done to further the scientific enterprise and not to benefit the individuals within those institutions
What did Mertons organised skepticism refer to?
Scientific research should be exposed to critical scrutiny before being accepted.
What is open science?
The conduct of research that allows other people to collaborate and contribute.
Everything is freely available in an online repository where anyone can freely access the information to collaborate
In the past, what was shared about the research?
Only the results and things that were right at the end
In the new models of research, what is shared about the experiment?
Every stage of the research process
Being open helps other scientists _____ your experiment
Replicate
What are the sources of error and bias that occur in the research planning phase?
What are the open science solutions?
Sources of error and bias:
Quantity over Quality
Emphasis on smallest publishable unit
Underpowered designs
Open Science Solutions:
Changing norms and reward structures
Technological solutions
What are the sources of error and bias that occur in the data collection and methods reporting phase?
What are the open science solutions?
- Optional stopping
- Failure to disclose all treatments, covariates and dependent measures
* Solutions*:
21-word solution
Open materials
Open workflow
Pre-registration
What are the sources of error and bias that occur in the data analysis and results reporting phase?
What are the open science solutions?
- HARKing
- p-hacking
- Flexible data cleaning and analytic choices
* Solutions:* - Open data and code
- Pre-registration
- Publishing replications
- Improved statistical training and re-training
- DDistinguishing exploratory from confirmatory results
What are the sources of error and bias that occur in the publication process phase?
What are the open science solutions?
- Sexy is better than solid
- Emphasis on p < .05
- Overclaiming
- Tidy stories
- Lack of publication of replications
* Solutions* - Open peer review
Post-publication peer review
Pre-registered reports
Alternatives to tradutional manuscrips
Preprints and new media
Tolerance of ‘messy/imperfect findings’