L7 Flashcards

1
Q

What is required in order for us to have more reproducible science?

A

Openness and sharing as many parts of these features of the research process as possible

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

How would an open culture help reproducibility?

A

If we change the incentives for getting published we can change the the incentive to the quality of the research rather than the finding of significant results.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What were Robert Mertons 4 norms for how science should be done?

A

Communalism

Universalism

Disinterestedness

Organised skepticism

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What did Mertons communalism refer to?

A

Openness and transparency in research

Science should be shared

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What did Mertons universalism refer to?

A

Science should be independent of sociopolitical status

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What did Mertons disinterestedness refer to?

A

Science should be done to further the scientific enterprise and not to benefit the individuals within those institutions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What did Mertons organised skepticism refer to?

A

Scientific research should be exposed to critical scrutiny before being accepted.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What is open science?

A

The conduct of research that allows other people to collaborate and contribute.

Everything is freely available in an online repository where anyone can freely access the information to collaborate

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

In the past, what was shared about the research?

A

Only the results and things that were right at the end

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

In the new models of research, what is shared about the experiment?

A

Every stage of the research process

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Being open helps other scientists _____ your experiment

A

Replicate

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What are the sources of error and bias that occur in the research planning phase?

What are the open science solutions?

A

Sources of error and bias:

Quantity over Quality

Emphasis on smallest publishable unit

Underpowered designs

Open Science Solutions:

Changing norms and reward structures

Technological solutions

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What are the sources of error and bias that occur in the data collection and methods reporting phase?

What are the open science solutions?

A
  1. Optional stopping
  2. Failure to disclose all treatments, covariates and dependent measures
    * Solutions*:

21-word solution

Open materials

Open workflow

Pre-registration

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What are the sources of error and bias that occur in the data analysis and results reporting phase?

What are the open science solutions?

A
  1. HARKing
  2. p-hacking
  3. Flexible data cleaning and analytic choices
    * Solutions:*
  4. Open data and code
  5. Pre-registration
  6. Publishing replications
  7. Improved statistical training and re-training
  8. DDistinguishing exploratory from confirmatory results
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What are the sources of error and bias that occur in the publication process phase?

What are the open science solutions?

A
  1. Sexy is better than solid
  2. Emphasis on p < .05
  3. Overclaiming
  4. Tidy stories
  5. Lack of publication of replications
    * Solutions*
  6. Open peer review

Post-publication peer review

Pre-registered reports

Alternatives to tradutional manuscrips

Preprints and new media

Tolerance of ‘messy/imperfect findings’

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What are the sources of error and bias that occur in the storing and archiving phase?

What are the open science solutions?

A
  1. File drawer problem
  2. Information loss creates bias in meta-analyses
    * Sources:*
  3. Open materials, data and workflow
  4. Rewards for sharing innovations in curation and indexing
17
Q

What are the ethical implications of open research?

A

Can we share without identifying participants?

Legal issues with sharing material?

18
Q

What is the open practice of preregistration?

A

Stating as clearly and specifically as possible what you plan to do, and how, before doing it, in a manner that is verifiable by others

19
Q

How would we go about preregistering out experiment?

A

Sharing the hypothesis and design before we start analysing the data

shows the difference between confirmatory and exploratory analysis

20
Q

What is the open practice of open source?

A

Opting for software and tools that are free to use so that others can use them to reproduce results

21
Q

What is the open practice of open data?

A

Sharing raw data files and summary data failes in a manner that others can reproduce the results

22
Q

What is the open practice of open materials?

A

Sharing materials and measures in a manner that enables others to directly replicate a study.

23
Q

What is the open practice of open data analysis?

A

Sharing data analytic scripts or code so that others can reproduce the results.

24
Q

What is the open practice of open peer review?

A

Making reviews publicly available for others to evaluate criticisms and responses.

25
Q

What is the open practice of open access publishing?

A

Making research outputs freely available to the public and other researchers

26
Q

Name the 6 open science practices.

A