Lecture 8

Question 1

What is P-Hacking?

Answer 1

Deliberate and selective reporting of significant P-values-deliberate abuse of Df. Can include correlational methods and overfitting of data

Question 2

What percentage of researchers admit to questionable practices?

Answer 2

10%

Question 3

What percentage of researchers admit to dropping data based on a “gut feeling”?

Answer 3

15%

Question 4

What percentage of researchers allowed industry funders to write their first draft?

Answer 4

30%

Question 5

What percentage of researchers allowed an industry funder to decide when study is terminated?

Answer 5

34%

Question 6

What percentage of researchers know someone who has failed to report data that contradicts their hypothesis?

Answer 6

69%

Question 7

What percentage of researchers know someone who has chosen different statistical techniques that decreases the P-value?

Answer 7

46%

Question 8

What percentage of researchers know someone who has reported only significant results?

Answer 8

46%

Question 9

What percentage of researchers know someone who has excluded data based on a “gut feeling”

Answer 9

20%

Question 10

What happens when we look at the predicted P-values graph?

Answer 10

The number of studies reporting something that just barely makes P < 0.05 is way higher than predicted. There are fewer P values over 0.05 than expected as well. Suggests that researchers are messing with data to put it in the “just less than 0.05 section”

Question 11

What is HARKing?

Answer 11

Hypothesizing After Results ae Known. Present a hypothesis that was made after data collection as though it was made beforehand. Collect data on a bunch of variables and see what sticks

Question 12

What are some solutions to the problems with data collection?

Answer 12

1) Effect sizes
2) Transparency
3) Providing all materials needed for replication
4) Make raw data available

Question 13

How can effect size help us?

Answer 13

Effect sizes tell you the size of the effect in an objective and standardized way-independent of sample size, allows for comparison across studies (even one’s with different measures and sample sizes)

Question 14

What is Cohen’s d?

Answer 14

Measure of effect size wherein the difference between 2 comparison means is divided by the pooled SD. Tells us how overlapped our two populations are. Effect size will remain the same regardless of sample size

Question 15

What is Pearsons r?

Answer 15

Correlation coefficient that tells us how strongy related 2 variables are-only used if one of the variables is categorical with 2 categories.

Question 16

What are some ways we can be more transparent about data collection?

Answer 16

Write hypotheses in advance on a website that will publish and timestamp and date it, propose line of research to journal who will agree to publish regardless of findings