Chapter 14- Replication, Transparency, and Real World Importance

Question 1

Replicable

Answer 1

Whether researchers will get the same results if they conduct the same study again. This is part of interrogating statistical validity- we ask about effect size, confidence interval, and what else is known from other studies

Question 2

Direct replication

Answer 2

Researchers repeat an original study as closely as they can to see whether the effect is the same in the newly collected data. If there are any threats to internal validity or other flaws in the original study, these flaws will be repeated. Also, the theory will not be tested in a new context

Question 3

Conceptual replication

Answer 3

Researchers explore the same research question but use different procedures. The conceptual variables in the study are the same, but the procedures for operationalizing them are different.

Question 4

Replication plus extension

Answer 4

Researchers replicate their original experiment and add variables to test additional questions

Question 5

Why might a study not be replicable?

Answer 5

It could be an issue with the replication study- there are differences with sample, materials, etc even in direct replications. There could also have been a problem with the original study.

Question 6

Scientific literature

Answer 6

Consists of a series of related studies conducted by various researchers that have tested similar variables. You might hear about the literature on alcohol and aggression, etc

Question 7

Meta-analysis

Answer 7

A way of mathematically averaging the results of all the studies that have tested the same variables to see what conclusion that whole body of evidence supports

Question 8

File drawer problem

Answer 8

The idea that a meta-analysis might be overestimating the true size of the effect because negligible or opposite effects have not been included in the collection process.

Question 9

Harking

Answer 9

Creating an after the fact hypothesis about a research result, so it appears the results were predicted all along. This misleads readers about the strength of the evidence

Question 10

p-hacking

Answer 10

Researchers might remove outliers from the data, compute scores several different ways, or run a few different types of statistics. This is called p hacking because the goal is to get the p value under .05. This might not be done intentionally, but biases creep in.

Question 11

Open science

Answer 11

The practice of sharing one’s data and materials freely so others can collaborate, use, and verify the results

Question 12

Open data

Answer 12

Psychologists provide their full data set, so other researchers can reproduce the statistical results or conduct new analyses on it.

Question 13

Open materials

Answer 13

Psychologists provide their study’s full set of measures and manipulations so others can conduct replication studies more easily. All conditions and measured variables are reported, not just the ones that worked.

Question 14

Preregistration

Answer 14

When scientists publish their study’s method, hypotheses, or statistical analyses in advance of data collection.

Question 15

Which validity does replicability help interrogate?

Answer 15

External validity- especially when it’s a conceptual replication or replication plus extension study because they demonstrate that the results can be generalized to other settings

Question 16

How is a study’s generalizability to other people assessed?

Answer 16

You would ask how the participants were obtained. If a convenience sample is used, you can’t be sure of generalizability. If intended to generalize to a certain population, the researchers must draw a probability sample from that population.

Question 17

Ecological validity

Answer 17

A study’s similarity to real world contexts

Question 18

Theory testing mode

Answer 18

When researchers are designing correlational or experimental research to investigate support for a theory. In theory testing mode, external validity often matters less than internal validity. An example is Harlow’s contact comfort experiment. The majority of studies in psychology are theory testing.

Question 19

Generalization mode

Answer 19

When researchers want to generalize the findings from the sample in a previous study to a larger population. Generalization mode is more concerned with external validity, and researchers are careful to use probability samples.

Question 20

Cultural psychology

Answer 20

A subdiscipline of psychology focusing on how cultural contexts shape the way a person thinks, feels, and behaves

Question 21

Muller-Lyer illusion

Answer 21

Two lines are actually the same length, but one appears longer. Almost all North Americans and Europeans fall for it, but many people around the world don’t.

Question 22

Field setting

Answer 22

When research takes place in the real world

Question 23

Experimental realism

Answer 23

When lab experiments create situations in which people experience authentic emotions, motivations, and behaviors