Questionable Research Practices (QRP's)

Question 1

jelly beans

What does P-Hacking involve?

Answer 1

Carrying out many statistical tests in the hope that one of them is significant, but not reporting this (1 in 20m chance of false positive)
essentially making data look significant when it is not and then covering it up

Question 2

Optional stopping involves _________________________-as you recruit and __________ when the data looks good.

Answer 2

Optional stopping = carrying out data analysis as you recruit and stopping when the data looks good

Question 3

Optional stopping is also referred to as
A Data blanketing
B Data snowing
C Data Raining
D Data peeking

Answer 3

D Data peeking

Question 4

Instead of optional stopping, what 2 alternative practices are acceptable? Explain them

Answer 4

Assessing data quality - if removing noisy data outliers then will have to include more
Sequential analysis - using a correction to the statistic, with this decision being made earlier

Question 5

What is uncorrected multiple testing?

Answer 5

Carrying out many tests/ measures but only reporting the most significant, leaving out many measures

Question 6

Researcher degrees of freedom refers to the __________ the researcher has in __________ design and _____ analysis. However this may mean that _________ researchers will produce a _________ outcome from the _____ results and methods.

Answer 6

Researcher degrees of freedom = aspects of design and data analysis that researcher has flexibility with
implies that different researchers will have a different outcome given different results

Question 7

What is the main implication of researcher degrees of freedom?

Answer 7

researchers have multiple options for analysis and may pick the best one that suits them

Question 8

HARKing is __________ _________ __ ________ ____ ______

Answer 8

Harking = hypothesizing after the results are known

(and presenting them as a priori hypotheses

Question 9

The file drawer problem is a form of _________ bias. It is the idea that that _______ results are more likely to be _______ than _______ results. Therefore many studies go __________, and will not be used to guide future _________. This leads to a _________ crisis

Answer 9

The file drawer problem is a publication bias where positive results more favorable than negative results
many studies go unpublished and cannot be used to guide future research
causes replication crisis

Question 10

What 2 types of registration avoid QRPs?

Answer 10

pre-registration report
Registered reports

Question 11

What is pre registration?

Answer 11

posting a time stamped plan of the methods, data plan and hypotheses before data analysis

Question 12

Registered reports are good as you can receive ______ at each stage and make __________. However this is __________ and ______ ________.

Answer 12

Registered reports - can receive feedback stage by stage of the study
however, this is inefficient and time consuming