Module 2: Methods and Bias Flashcards
cross-cultural validation studies
the valididty (= accuracy) and reliablility (= consistency) of a scale, test, or measure originally developed in one culture examined across different cultures
- aim is to check whether the measure is equivalent across cultures
indigenous cultural studies
these allow for in-depth analyses into a single culture
- the insights from these studies can then be compared across cultures
cross-cultural comparison studies
psychological constructs are compared between participants from 2 or more cultures
exploratory vs hypothesis testing
exploratory studies examine whether cultural differences or similarities exist
- hypothesis testing studies examine why cultural differences exist
presence vs absence of contextual factors
contextual factors are variables that can explain cross-cultural differences
structure vs level oriented
structure-oriented studies compare constructs, measures, or relationships with other constructs across cultures
- level-oriented studies compare differences between average scores across cultures
individual vs ecological (cultural) measurement level
in individual-level studies, the unit of analysis is data from individuals
- in ecological-level studies, the unit of analysis is date from countries or cultures
cannot assume isomorphism
we cannot assume that a relationship or association between variables is the same at every level
Simpsons Paradox
where there is a relationship in one life that does not exist in another
avoid ecological fallacy
occurs when country-level data are used to make statements about individuals of that country
avoid dichotomies
a division or contrast between 2 things that are, or are being presented as being opposed or entirely different
- individualist OR collectivist
- independent OR interdependent
avoid cultural essentialism
culture is believed to be a central aspect of someone’s personality that defines who they are
linkage studies
studies that attempt to discover a linkage between psychological and cultural variables
- unpacking studies
- experiments
unpacking studies
aim to explore and identify the specific cultural factors or mechanisms (like values, norms, or practices) that explain cultural differences in behavior, rather than simply noting that differences exist
- also called context variables (operationalize cultural constructs)
Poorting (2016, 2022)
compared unpacking studies to peeling an onion
- we cannot explain behavior by culture alone, but must ‘peel’ it layer by layer
- sees culture as a place holder for that which we do not know yet
experiments
research in which one variable is manipulated to observe its effect on the other variable
- 2 types: priming and behavioral
priming experiments
manipulating the participant’s train of thought to observe behavioral change
behavioral experiments
manipulating the environment to observe behavioral change
bias
a systematic error that endangers the comparability of data across cultures and groups
construct bias
the measured construct is not identical across cultures/groups
model bias
does the theoretical framework of whatever it is we are measuring translate in the same way across cultures?
sample bias
cross-cultural variation in sample characteristics, such as differences in gender, education, SES, or urbanization
instrument bias
the measurement instrument might be biased
stimulus familiarity
the extent to which participants are familiar with the stimuli, can have an enormous impact
response styles
systematic tendency to use certain categories of response
administration bias
administration conditions, ambiguous instructions, the interaction between administrator and respondent, or communication problems
interpretational bias
interpreting data based on one’s own culture, without considering that of another
- creates misinterpretation
Poortinga’s 4 ways to deal with interpretation non-equivalence
- ignoring
- stopping the comparison
- interpreting the non-equivalence
- reducing the non-equivalence
linguistic/item bias
cross-cultural research is done in many countries, speaking many languages, therefore it is a concern whether or not terms in a language have the same meaning when translated into other languages
back translation
helps reduce linguistic bias by translating a piece of text from language 1 to language 2, then translating it back to language 1
equivalence
the level of comparability across cultures and groups
- related to the measurement level at which scores obtained in different cultures can be compared
- e.g. kilometers vs miles can be converted into one another
construct equivalence
no construct bias is present
- we are measuring the same thing across cultures
measurement unit equivalence
with conversion, we can obtain equivalent scores
full score equivalence
no bias present
- this is the goal, but not always obtainable
minimization of bias
- design
- implementation
- analysis
design
- “how can i make my study culturally appropriate?”
- choice of instruments is important
- instruments can be adopted, adapted to the context, or assembled
implementation
“how can i conduct my study in a culturally appropriate way?”
- pilot studies, standard protocols, and clear instructions are important
analysis
“do my items behave differently?”
- exploratory/confirmatory factor analysis and differential item functioning analysis can be used
qualitative research
done in a natural setting or in the field
- help to gather information about a culture we are dealing with for the first time, build theoretical models, and generate hypotheses
- depends on interpretation, and it is difficult to formalize procedures
- a lot of room for bias
quantitative research
independent and dependent variables are used
- useful for hypothesis testing
- often difficult to control for the variables
- post hoc cultural interpretations are very susceptible to bias
