Articles

Question 1

Name 4 common degrees of freedom when conducting research

Answer 1

Four common degrees of freedom are choosing sample size (1), using covariates (2), choosing among dependent variables (3) and reporting subsets of experimental conditions (4).

Question 2

What are the six guidelines for preventing the increased rate of false positives?

Answer 2

1. Authors must decide the rule for terminating data collection before data collection begins and report this rule in the article.
2. Authors must collect at least 20 observations per cell or else provide a compelling cost-of-data collection justification.
3. Authors must list all variables collected in a study.
4. Authors must report all experimental conditions, including failed manipulations
5. If observations are eliminated, authors must also note what the statistical results are if those observations are included.
6. If an analysis includes a covariate, authors must report the statistical results of analysis without the covariate.

Question 3

What are the four guidelines of reviewers to prevent the increased rate of false positives?

Answer 3

1. Reviewers should ensure that authors follow the requirements.
2. Reviewers should be more tolerant of imperfection in results.
3. Reviewers should require authors to demonstrate that their results do not hinge on arbitrary
   analytic decisions.
4. If justifications of data collection or analysis are not compelling, reviewers should require the
   authors to conduct an exact replication.

Question 4

What is meant by hypothesis myopia

Answer 4

Collecting evidence to support a hypothesis, not against it and ignoring other explanations

Question 5

How can you debias hypothesis myopia?

Answer 5

Devil’s advocacy

Question 6

What is meant by the texas sharpshooter fallacy?

Answer 6

seizing on random patterns in the data and mistaking them for interesting findings

Question 7

How can you combat texas sharpshootin’

Answer 7

Pre committing by publicly declaring a data collection and analysis plan before starting a study

Question 8

What is meant by asymmetric attention?

Answer 8

Rigorously checking unexpected results, but giving unexpected ones a free pass

Question 9

How is asymmetric attention avoided?

Answer 9

Invite academic adversaries to collaberate

Question 10

What is meant by just-so storytelling?

Answer 10

Finding stories after the fact to rationalise whatever the results turn out to be

Question 11

How do you avoid just-so storytelling?

Answer 11

Blind data analysis- analyse data that look real, but are not exactly what you left behind, then lift the blind

Question 12

What is meant by yarking?

Answer 12

justifying after results are known-rationalising why results should have come up a certain way but did not (not exactly significant but…)