L7 Flashcards
What is required in order for us to have more reproducible science?
Openness and sharing as many parts of these features of the research process as possible
How would an open culture help reproducibility?
If we change the incentives for getting published we can change the the incentive to the quality of the research rather than the finding of significant results.
What were Robert Mertons 4 norms for how science should be done?
Communalism
Universalism
Disinterestedness
Organised skepticism
What did Mertons communalism refer to?
Openness and transparency in research
Science should be shared
What did Mertons universalism refer to?
Science should be independent of sociopolitical status
What did Mertons disinterestedness refer to?
Science should be done to further the scientific enterprise and not to benefit the individuals within those institutions
What did Mertons organised skepticism refer to?
Scientific research should be exposed to critical scrutiny before being accepted.
What is open science?
The conduct of research that allows other people to collaborate and contribute.
Everything is freely available in an online repository where anyone can freely access the information to collaborate
In the past, what was shared about the research?
Only the results and things that were right at the end
In the new models of research, what is shared about the experiment?
Every stage of the research process
Being open helps other scientists _____ your experiment
Replicate
What are the sources of error and bias that occur in the research planning phase?
What are the open science solutions?
Sources of error and bias:
Quantity over Quality
Emphasis on smallest publishable unit
Underpowered designs
Open Science Solutions:
Changing norms and reward structures
Technological solutions
What are the sources of error and bias that occur in the data collection and methods reporting phase?
What are the open science solutions?
- Optional stopping
- Failure to disclose all treatments, covariates and dependent measures
* Solutions*:
21-word solution
Open materials
Open workflow
Pre-registration
What are the sources of error and bias that occur in the data analysis and results reporting phase?
What are the open science solutions?
- HARKing
- p-hacking
- Flexible data cleaning and analytic choices
* Solutions:* - Open data and code
- Pre-registration
- Publishing replications
- Improved statistical training and re-training
- DDistinguishing exploratory from confirmatory results
What are the sources of error and bias that occur in the publication process phase?
What are the open science solutions?
- Sexy is better than solid
- Emphasis on p < .05
- Overclaiming
- Tidy stories
- Lack of publication of replications
* Solutions* - Open peer review
Post-publication peer review
Pre-registered reports
Alternatives to tradutional manuscrips
Preprints and new media
Tolerance of ‘messy/imperfect findings’
What are the sources of error and bias that occur in the storing and archiving phase?
What are the open science solutions?
- File drawer problem
- Information loss creates bias in meta-analyses
* Sources:* - Open materials, data and workflow
- Rewards for sharing innovations in curation and indexing
What are the ethical implications of open research?
Can we share without identifying participants?
Legal issues with sharing material?
What is the open practice of preregistration?
Stating as clearly and specifically as possible what you plan to do, and how, before doing it, in a manner that is verifiable by others
How would we go about preregistering out experiment?
Sharing the hypothesis and design before we start analysing the data
shows the difference between confirmatory and exploratory analysis
What is the open practice of open source?
Opting for software and tools that are free to use so that others can use them to reproduce results
What is the open practice of open data?
Sharing raw data files and summary data failes in a manner that others can reproduce the results
What is the open practice of open materials?
Sharing materials and measures in a manner that enables others to directly replicate a study.
What is the open practice of open data analysis?
Sharing data analytic scripts or code so that others can reproduce the results.
What is the open practice of open peer review?
Making reviews publicly available for others to evaluate criticisms and responses.
What is the open practice of open access publishing?
Making research outputs freely available to the public and other researchers
Name the 6 open science practices.
