Open science Flashcards
What term is used to describe a methodological crisis in which the results of the studies were unable to be replicated/reproduced when tested again?
Replication crisis
Describe the Replication Crisis
A methodological crisis in which the results of the studies were unable to be replicated/reproduced when tested again
What happens when an experiment faces a replication crisis?
Inaccurate/not trustworthy = affects several scientific disciplines
What influences a replication crisis in an experiment?
- Researchers feel pressured to publish a novel of sensational findings (their work won’t be published if the study isn’t unique and interesting)
- Sometimes researchers manipulate their results/hypothesis if they don’t get the results they wanted; in order for their work to appear as sensational and will later be published
Match the items.
a. research focused on testing hypotheses and research questions
b. research that checks whether using the same experimental methods produces the same results
c. research that checks whether using the same analyses and the same data produces the same results
d. research focused on generating hypotheses and research questions
- Replication research
- Reproduction research
- Confirmatory research
- Exploratory research
- B
- C
- A
- D
Which of the following is NOT one of the basic principles of Open Science?
a. Open Access
b. Replication Research
c. Teaching Open Science
d. Exploratory Research
D
For how many years does the APA require researchers to make data available post publication?
a. 0 b. 1 c. 5 d. 10
C
Pre-registration refers to…
a. Making research questions and planned analyses clear prior to data collection
b. Making sure that you keep datasets clearly annotated for future reference
c. Making sure you get an independent researcher to check all analyses performed
d. Making data, code and methods clear and available for wider public use
A
What is an open science?
“Transparent” research
What research is considered “Transparent” research?
When the research has more processes involved in coming up with the research (e.g. a lot of thought went into thinking of the hypothesis, methods etc)
What are the 2 most important aspects of open research?
1) Must have good reproducibility
2) Must have good replicability
Define reproducibility
Getting the same results if we do the experiment ourselves using the SAME DATA and perform the SAME ANALYSES as the study we are trying to reproduce
Define replicability
Getting the same results if we do the experiment ourselves following the SAME METHODS as the study we are trying to reproduce
What term is used to describe this:
Getting the same results if we do the experiment ourselves following the SAME METHODS as the study we are trying to reproduce
Replicability = SAME METHODS
What term is used to describe this:
Getting the same results if we do the experiment ourselves using the SAME DATA and perform the SAME ANALYSES as the study we are trying to reproduce
Reproducibility = SAME ANALYSES and SAME DATA
What is the term used to refer to unrestricted public access to research?
Open access
Define open access
The unrestricted public access to research
What are the benefits of open access? List 4 benefits
1) Accumulation of knowledge
2) Increased citation of work (work will be more likely to be recognised)
3) More media coverage
4) Support meta-research practice (reanalyse data with data received from other published studies; larger data set)
What must researchers share if they want to publish their work, according to the APA guidelines? List 3 requirements
1) Data
2) Protocols
3) Code for experiments and analyses
What are the benefits of open data, materials/protocols and code?
1) Verification = results can be checked to minimise errors and biases
2) Analytic reproducibility = checking which steps were taken to prepare and analyse the data
Define Verification
Verification = results can be checked to minimise errors and biases
Define Analytic reproducibility
Analytic reproducibility = checking which steps were taken to prepare and analyse the data
What term is used to describe when results can be checked to minimise errors and biases?
Verification
What term is used to describe checking which steps were taken to prepare and analyse the data?
Analytic reproducibility
What are reproducible analyses?
- Data must be organised
- The study must provide materials to enable others to generate the same exact results as those reported
How do you provide materials to enable others to generate the same exact results as those reported?
Adequate documentation must be kept such as:
1) Provide clear annotations of what the documents are
2) Store original data files separately
3) Record all steps of data processing
4) Use open-source software where possible
When wanting to publish studies, what do researchers need to do in order to be considered for publishing (to ensure ethical experiments and results)?
Pre registration
What is preregistration?
When researchers are encouraged to submit their plans for their specific research questions that they wish to address and the analyses they will conduct prior to data collection
What is it called when researchers are encouraged to submit their plans for their specific research questions that they wish to address and the analyses they will conduct prior to data collection?
Preregistration
What are the 2 types of research?
1) Confirmatory research
2) Exploratory research
What is Confirmatory research?
Research that focuses on CONFIRMING hypotheses or research question
What is Exploratory research?
Research that focuses on GENERATING hypotheses or research questions
What type of research focuses on GENERATING hypotheses or research questions
Exploratory research
Why type of research focuses on CONFIRMING hypotheses or research question?
Confirmatory research
Which type of research (confirmatory/exploratory) can be preregistered?
Confirmatory
Why can’t exploratory research be preregistered?
Because it does not have a set, structured plan unlike confirmatory research (exploratory research relies on exploring data and conducting as many analyses as possible to create a hypothesis)
What is replication research?
The process of repeating research to verify findings (using the same methods)
What is the purpose of preregistration?
Reduce risks of false positive results
What will replication research do?
Enables confidence in results and helps build theories
What are the 2 types of replication research?
1) Direct replications
2) Conceptual replications
Define Direct replications
Repeat everything done in the original study exactly (reproduce the elements that produced the original findings)
e.g. Use the same number of participants, exact same stimuli and the same number of trials as the original study
Define Conceptual replications
Repeat the original study but change at least 1 aspect of it
e.g. Use a different sample or a different stimuli to the original study
What type of replication involves repeating everything done in the original study exactly as it was done (reproducing the elements that produced the original findings)?
Direct replication
What type of replication involves repeating the original study but changing at least 1 aspect of it?
Conceptual replication
What is the purpose of direct replications?
To assess whether similar findings are produced in subsequent attempts
(whether you get similar results to the original study if you use the same exact methodology over and over again)
What is the purpose of conceptual replications?
To assess whether similar findings are produced under different conditions (whether you get similar results to the original study if you change some parts of the study methodology)
Why do we teach open science?
1) To promote the best ethical practice
2) To increase critical evaluation of research
