Ch 14

Question 1

Replicable

Answer 1

the same results have actually been reproduced
- Not that the study could be replicated, but that the result of the study has been repeated

Question 2

types of replication studies

Answer 2

• Direct replication
• Conceptual replication
• Replication-plus-extension

Question 3

Direct replication (Exact replication) + risks

Answer 3

original study is repeated as closely as possible to determine if the original effect is found in the new data

• Risk of having the same threats to internal validity or construct validity in the original study - other types address this

Question 4

Conceptual replication:

Answer 4

same research question, different procedures- conceptual variables are the same, but the variables are operationalized differently

Question 5

Replication-plus-extension

Answer 5

Replication-plus-extension: researchers replicate their original experiment and add variables to test additional questions or to better understand the scope of the original effect

• Could add an additional level to an IV
• or add an additional IV

Question 6

Replication Crisis

Answer 6

Journals prefer to publish new research, no incentive to do direct replication studies

• Open Science Collaboration decided to attempt replication on larger scale, (39% was lowest rate)

Question 7

Why might replication studies fail?

Answer 7

• Contextually sensitive effects:
• Number of replication attempts:
• problems with original study

Question 8

(why might fail) Contextually sensitive effects:

Answer 8

Contextually sensitive effects:
- Measures and manipulations used in replications might not have same meanings as in original study- too sensitive, if replication context is different might fail

Question 9

(why might fail) Number of replication attempts:

Answer 9

• One replication- any single study has the potential to miss a true finding
• Many Labs Project- did up to 36 replications of each study + combined them (replication rate rose to 85%)

Question 10

(why might fail) problems with original study

Answer 10

Sample size-
- too small, extreme participants could’ve had disproportionate influence on means and pattern

Harking

P-hacking

Question 11

Harking

Answer 11

Hypothesizing after the results are known - usually due to a surprising result.
more likely due to chance, can’t be replicated (Type I errors more likely)

Question 12

P-hacking

Answer 12

• Researcher peaks at study’s results, if not significant may run a few more individuals, remove outliers, or run different type of analysis
• Called p-hacking because trying to get statistically significant reading- p of under .05

Question 13

improvements to scientific practice

Answer 13

• Larger sample sizes: Journals now require much larger samples for both original and replication studies
• Report all analyses and variables
• Open Science
• preregistration

Question 14

open science

Answer 14

Sharing ones data and materials freely so others can collaborate, use, and verify results

Open data- provides full data set so researchers can conduct new analyses on it, or reproduce results

Open materials- provides full set of measures and manipulations (ALL variables) so others can replicate study

Question 15

preregistration

Answer 15

Preregister study’s method, hypotheses, or statistical analyses online, before collecting data

• Discourages p-hacking

Question 16

meta-analysis

Answer 16

statistical analysis that gives a quantitative summary of a scientific literature

• Averages the results of all the different studies

Question 17

Scientific literature

Answer 17

Series of related studies conducted by different researchers who have tested similar variables

• A review article collects all the literature on a topic and reviews them together

Question 18

File drawer problem

Answer 18

since journals don’t publish null effects or opposite effects as often, a meta-analysis of published studies may overstate the size of an overall effect

• To counteract, researchers conducting meta-analyses should ask their colleagues for published and unpublished data

Question 19

probablility sample vs. convenience sample

Answer 19

A probability sample is intended to generalize to the population it was drawn from, but convenience sample may not generalize to the intended population

Question 20

external validity- what’s important?

Answer 20

External validity comes from how not how many- how obtained is more important

Just because a sample comes from a population doesn’t mean it generalizes to that population

Question 21

how can we see if a study generalizes to other settings?

Answer 21

Conceptual replication can show if a study generalizes to other settings

Question 22

Ecological validity(mundane realism)

Answer 22

aspect of external validity that asks if a study conducted in a lab generalizes to real-world settings

Question 23

Theory-testing mode:

Answer 23

typically used to test association or causal claims, to investigate support for a particular theory

• In theory testing mode, internal validity is more important than external validity

Question 24

Generalization mode:

Answer 24

where researchers want to generalize the findings of a previous study

• Important to use probability samples w/ diversity
• Applied research tends to be done in generalization mode

Question 25

what mode are frequency claims done in?

Answer 25

generalization mode

Question 26

what mode are association and causal claims done in?

Answer 26

Usually in theory testing mode, sometimes generalization

• Might first try it in theory testing mode, then shift into generalization mode

Question 27

Cultural Psychology:

Answer 27

type of psychology that focuses on cultural contexts and how they shape thoughts, emotions, and behavior

• Work in generalization mode, challenge researchers who don’t consider context and only work in theory-testing mode

Question 28

WEIRD

Answer 28

Western, Educated, Industrialized, Rich, and Democratic

WEIRD samples aren’t representative

Question 29

field setting

Answer 29

When a study takes place in the real world, it occurs in a field setting and has high external validity

Question 30

ecological validity

Answer 30

how similar a study’s manipulations or tasks are to situations participants might encounter in their lives

• field settings have high ecological validity
• important to consider in terms of generalizing to non-lab settings

Question 31

Experimental realism:

Answer 31

when lab experiments create situations in which people experience actual emotions, motivations, and behaviors

Laboratory research can be just as realistic as studies in the real world