Lecture 23 Flashcards
What are the 3 types of validity?
internal validity
construct validity
External validity
What is external validity?
External validity is the degree to which a pattern of results generalizes across:
People
Situations
Time
What do we call psychology populations?
Psychology populations are W.E.I.R.D
Western
Educated
Industrialized § Rich
Democratic
what percent of the population is Canada and the US, despite this, how many studies are conducted here?
less than 5%
Majority of studies conducted in America/Canada
§ Specifically subset sample within US/Can § What about the other 99%
What are some reasons that some of our best journals are even more WEIRD?
What can we infer form this?
Best journals are even more WEIRD! Example: Write a non-WEIRD paper…
Critiques from top journals:
“This is really great work, but it isn’t suited to our general audience”
“This would be interesting to a specialty journal that studies cross- cultural issues”
“This sample has a lot of international respondents, I’m concerned about the quality of these results”
What values can we infer?
WEIRD samples are what people want to read about
WEIRD samples are ‘culture-free,’ everyone else is culturally biased
WEIRD samples are more valuable
WEIRD research is more valuable
WEIRD samples are ‘normal’ and generalizable, other samples are not
What were the results of the UBC study investigating influence of parenting on decisions about marriage of north american and kurdish people.
Example: I wonder how much influence parents have on people’s decisions about marriage?
Method: UBC students self-reported how much influence parents had WEIRD participants
Non-WEIRD participants from Kurdistan
Results: Culture moderated how much influence parents had
Parents of WEIRD students had a pretty small influence
Parents of Kurdish students had a much stronger influence
Is the stroop colour word task similar across cultures?
yes!
What do we know about cognitive processes and culture?
Some cognitive processes are similar across cultures, but others are sensitive to early learning and experience
What are the 2 main well-known sample differences categories and specific implications?
Volunteers (vs. paid participants): § More social
More educated
More fluid intelligence
More sensitive to social approval
Sex of participants:
Differences in early learning
Differences in social roles, expectations
Differences in hormones, phenotypic expression
are research labs typically representative of the real world?
Research labs are often not representative of the ‘real world’ § Internal validity often requires artificiality
External validity is sacrificed, we know this and it is good!
Limitation: We don’t know if pattern of results will be similar outside the lab
Observational studies and field experiments can shed light on whether lab results are seen in the noisy ‘real-world’
Can experimental realism impact generalizability?
yes.
Experimental realism: Does the experiment psychologically involve participants?
If not, the psychological processes will differ
(unless we’re studying boredom??)
Is experimental realism a challenge? Why?
Weird flex, but OK…
Experimental Realism is a challenge:
Some are not psychologically involved in this study
#ScienceTerms “Key mashing” or “Mashers”
Psychological process is different from what we wish to model
Would people be psychologically involved if they encountered this situation outside the lab?
(Hell yes they would)
does mundane realism impact generalizability?
yes. Mundane realism: Does the experiment approximate the real- world?
§ If not, the psychological processes will again differ
First-person shooter task has been used with police officers § They show somewhat similar biases
They’re not “mashers”
Mundanerealismisstill low § Arousal is very different
Officers don’t work in 2-D
Real-world has consequences
What are some solutions to generalizing?
Factorial designs:
Compare WEIRD samples to non-WEIRD samples as an additional
factor within an experiment
Principled arguments:
Make a theoretically-informed argument why an effect should be sensitive or insensitive to culture
Meta-analysis:
Compile all past results on a given phenomenon and test whether
those results were dependent on a moderating factor
ManyLabs:
Massive experiments across many labs, cultures, samples
how to talk about generalization?
“I hypothesize that this effect won’t replicate in a non-WEIRD sample, because this psychological process is probably learned (rather than innate). Thus, WEIRD participants should have psychological process X, while non-WEIRD participants should have psychological process Y.”
