Ch 14 Flashcards

1
Q

Replicable

A

the same results have actually been reproduced
- Not that the study could be replicated, but that the result of the study has been repeated

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

types of replication studies

A
  • Direct replication
  • Conceptual replication
  • Replication-plus-extension
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Direct replication (Exact replication) + risks

A

original study is repeated as closely as possible to determine if the original effect is found in the new data

  • Risk of having the same threats to internal validity or construct validity in the original study - other types address this
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Conceptual replication:

A

same research question, different procedures- conceptual variables are the same, but the variables are operationalized differently

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Replication-plus-extension

A

Replication-plus-extension: researchers replicate their original experiment and add variables to test additional questions or to better understand the scope of the original effect

  • Could add an additional level to an IV
  • or add an additional IV
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Replication Crisis

A

Journals prefer to publish new research, no incentive to do direct replication studies

  • Open Science Collaboration decided to attempt replication on larger scale, (39% was lowest rate)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Why might replication studies fail?

A
  • Contextually sensitive effects:
  • Number of replication attempts:
  • problems with original study
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

(why might fail) Contextually sensitive effects:

A

Contextually sensitive effects:
- Measures and manipulations used in replications might not have same meanings as in original study- too sensitive, if replication context is different might fail

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

(why might fail) Number of replication attempts:

A
  • One replication- any single study has the potential to miss a true finding
  • Many Labs Project- did up to 36 replications of each study + combined them (replication rate rose to 85%)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

(why might fail) problems with original study

A

Sample size-
- too small, extreme participants could’ve had disproportionate influence on means and pattern

Harking

P-hacking

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Harking

A

Hypothesizing after the results are known - usually due to a surprising result.
more likely due to chance, can’t be replicated (Type I errors more likely)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

P-hacking

A
  • Researcher peaks at study’s results, if not significant may run a few more individuals, remove outliers, or run different type of analysis
  • Called p-hacking because trying to get statistically significant reading- p of under .05
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

improvements to scientific practice

A
  • Larger sample sizes: Journals now require much larger samples for both original and replication studies
  • Report all analyses and variables
  • Open Science
  • preregistration
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

open science

A

Sharing ones data and materials freely so others can collaborate, use, and verify results

Open data- provides full data set so researchers can conduct new analyses on it, or reproduce results

Open materials- provides full set of measures and manipulations (ALL variables) so others can replicate study

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

preregistration

A

Preregister study’s method, hypotheses, or statistical analyses online, before collecting data

  • Discourages p-hacking
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

meta-analysis

A

statistical analysis that gives a quantitative summary of a scientific literature

  • Averages the results of all the different studies
17
Q

Scientific literature

A

Series of related studies conducted by different researchers who have tested similar variables

  • A review article collects all the literature on a topic and reviews them together
18
Q

File drawer problem

A

since journals don’t publish null effects or opposite effects as often, a meta-analysis of published studies may overstate the size of an overall effect

  • To counteract, researchers conducting meta-analyses should ask their colleagues for published and unpublished data
19
Q

probablility sample vs. convenience sample

A

A probability sample is intended to generalize to the population it was drawn from, but convenience sample may not generalize to the intended population

20
Q

external validity- what’s important?

A

External validity comes from how not how many- how obtained is more important

Just because a sample comes from a population doesn’t mean it generalizes to that population

21
Q

how can we see if a study generalizes to other settings?

A

Conceptual replication can show if a study generalizes to other settings

22
Q

Ecological validity(mundane realism)

A

aspect of external validity that asks if a study conducted in a lab generalizes to real-world settings

23
Q

Theory-testing mode:

A

typically used to test association or causal claims, to investigate support for a particular theory

  • In theory testing mode, internal validity is more important than external validity
24
Q

Generalization mode:

A

where researchers want to generalize the findings of a previous study

  • Important to use probability samples w/ diversity
  • Applied research tends to be done in generalization mode
25
what mode are frequency claims done in?
generalization mode
26
what mode are association and causal claims done in?
Usually in theory testing mode, sometimes generalization - Might first try it in theory testing mode, then shift into generalization mode
27
Cultural Psychology:
type of psychology that focuses on cultural contexts and how they shape thoughts, emotions, and behavior - Work in generalization mode, challenge researchers who don’t consider context and only work in theory-testing mode
28
WEIRD
Western, Educated, Industrialized, Rich, and Democratic WEIRD samples aren’t representative
29
field setting
When a study takes place in the real world, it occurs in a field setting and has high external validity
30
ecological validity
how similar a study’s manipulations or tasks are to situations participants might encounter in their lives - field settings have high ecological validity - important to consider in terms of generalizing to non-lab settings
31
Experimental realism:
when lab experiments create situations in which people experience actual emotions, motivations, and behaviors Laboratory research can be just as realistic as studies in the real world