Chapter 14: Replication, transparency, and real world importance

Question 1

direct replication

Answer 1

researchers repeat the original study as closely as possible to see whether the original effect shows up in the newly collected data

Question 2

conceptual replication

Answer 2

researchers examine the same research question but use different procedures for operationalizing the variables

Question 3

replication-plus-extension study

Answer 3

researchers replicate their original study but add variables or conditions that test additional questions

Question 4

meta-analyses

Answer 4

a way of mathematically averaging the effect sizes of all the studies that have tested the same variables to see what conclusions the whole body of evidence supports

Question 5

file drawer problem

Answer 5

a problem relating to literature reviews and meta-analyses based only on published literature, which might overestimate the support for a theory because studies finding null effects are less likely to be published than studies finding significant results, and therefore less likely to be included in such reviews

Question 6

underreporting null findings

Answer 6

when researchers report only the strong effects of their results and not the weak ones

Question 7

harking

Answer 7

hypothesizing after the results are known

Question 8

p-hacking

Answer 8

a family of questionable data analysis techniques such as adding participants after the results are initially analyzed, looking for outliers, or trying new analysis in order to obtain a p-value of just under 0.05

Question 9

open science

Answer 9

an open practice of sharing one’s data, hypotheses, and materials freely so that others can collaborate, use, and verify the results

Question 10

open materials

Answer 10

providing their full set of measures and manipulations on the internet so that others can see the full design or conduct replication studies

Question 11

preregistration

Answer 11

before collecting any data, the researcher has publicly stated what the study’s outcome is expected to be

Question 12

ecological validity

Answer 12

the extent to which the tasks and manipulations of a study are similar to real-world contexts

Question 13

theory testing mode

Answer 13

researchers are designing correlational or experimental research to investigate support for a theory

Question 14

generalization mode

Answer 14

the intent of researchers to generalize the findings from the samples and procedures in their study to other populations and contexts

Question 15

field setting

Answer 15

a real-world setting for a research study

Question 16

experimental realism

Answer 16

the extent to which a lab experiment is designed so that participants experience authentic emotions, motivations, and behavior

Question 17

publication bias

Answer 17

the phenomenon that statistically significant findings have a higher probability of being published than nonsignificant findings