Chapter 2 (Unit 3) - Research Methods Flashcards
WHY TALK ABOUT
RESEARCH METHODS? LIMITATIONS OF PERSONAL EXPERIENCE
- A basic understanding of research methods is critical to consuming relationship research responsibly
- There is a lot of personal opinion—rather than scientific fact—competing for your attention
- Will encounter a lot of conflicting advice—
how to sift through it? - Personal experience can be valuable source of insight & inspiration
- But subject to several limitations
– Confirmation bias = tendency to seek out, pay attention to, and believe only
evidence that supports what we are already confident we know
– Our perspective is unique and limited
Experience has no control group
Often lack awareness of the factors driving our reactions and behaviours
*Too quick to see causal relationships where there may be none - Just because two things co-occur, doesn’t mean one causes the other
THE SCIENTIFIC APPROACH
* Thus, our lay theories are not always correct
* Relationship scientists examine questions systematically by collecting data
and testing their hypotheses while taking steps to reduce bias, subjectivity,
and rule out alternate explanations
CAN LOVE BE STUDIED?
* Studying love is not impossible, but it is challenging
– Things research scientists are interested in—like love, commitment,
satisfaction, trust, attraction—are all very abstract
– How we operationalize these psychological constructs is key
OPERATIONALIZING OUR CONSTRUCTS
- Can’t measure these psychological constructs directly, only their operationalizations
– E.g., attraction—could look at: - Behavioural manifestations (smiling,
exchanging phone numbers) - Self-reported ratings of attraction
- Physiological measures (heart rate, brain
activity) - Need to carefully consider whether we are really measuring what we think we are measuring
CONSTRUCT VALIDITY
* Construct validity = extent to which an
operationalization adequately captures
the psychological construct of interest
– “Are you measuring what you think
you’re measuring?”
* Should be well grounded in existing
theoretical conceptions of the construct
* Much harder than you might think!
DIFFERENT TYPES OF MEASURES: SELF-REPORT
- Simply ask participants to describe their
thoughts, feelings, & behaviours - Includes fixed-response questionnaires
and open-ended questions
– Fixed-response questionnaire = specific
set of questions and possible responses
predetermined by the researchers - E.g., The Love Scale (Rubin, 1970)
QUALITATIVE RESEARCH
* Open-ended questions = participant gives any answer that comes to mind
- Helpful when studying something we don’t know much about yet
- A way of gathering information to generate more specific questions later
on
* Qualitative research: methodological approach relying primarily on openended questions
– Examine the broader themes that emerge from participants’ responses
(content analysis)
SELF
-REPORT: PROS & CONS
Pros
* Inexpensive and easy to
administer, no special
equipment required – Means that we can
recruit more participants
* Allow us to “get inside
people’s heads”
Cons
* Difficulties with self-awareness and recall
* Social desirability bias (desire to be seen in a positive light)
– Relationship researchers deal with many sensitive topics
- E.g., higher reported rates of infidelity in online surveys vs. face-to-face
interviews (Whisman & Snyder, 2007)
* Participants may not interpret questions in the way you intended (ex: what is sex)
Now, imagine how much variation there might be in the way people interpret
questions about arguments, love, support, etc.!
Researchers must carefully define crucial terms on questionnaires
Good idea to talk to your participants during debriefing to better understand
their experience of the study
SENTIMENT OVERRIDE
(WEISS, 1984)
* What if we want to ask participants about their partner’s behaviours?
* Sentiment override = global beliefs about the partner/relationship may
colour perception and memory
- E.g., “how many times did your partner kiss you yesterday?”
– “I don’t remember, but he loves me and we have a good relationship, so
must have been a lot”
DIFFERENT TYPES OF MEASURES: BEHAVIOURAL OBSERVATION
- Gather data about relationship events without having to ask people who are experiencing those events directly
– E.g., how do people in happy and distressed relationships differ in the way they behave towards each other? - Train observers to watch & code recordings of participant behaviour
- Can observe people:
– In the lab
– At home
– Anywhere! (e.g., speed dating events, airports) - Need to decide which behaviours to observe
- Coders must agree about what constitutes incidents of a given behaviour
category
– Interrater reliability = extent to which coders agree on whether a specific
behaviour has or has not occurred
– A lot of behaviours require interpretation - E.g., lighthearted joke vs. hostile sarcasm
- May be easier to agree on a super concrete behaviour, but lose the broader
meaning of the behaviour (“can’t see the forest through the trees”)
– Requires extensive training
Pros
* Directly assess behaviours of interest
* Don’t have to rely on faulty memories
* Avoid social desirability bias
Cons
* Expensive, time and labour-intensive
* Reactivity = a change in behaviour caused
by the knowledge one is being observed
DIFFERENT TYPES OF MEASURES: INDIRECT MEASURES
- Designed to avoid reactivity and social desirability
- E.g., reaction time = time it takes to respond a stimulus on screen
– Can be used to assess implicit attitudes = the automatic tendency to
associate a given stimulus with positive or negative feelings
“THE NEWLYWED GAME”
(MCNULTY ET AL., 2013)
Index of automatic attitudes = negative word RT – positive word RT
Higher score = more positive attitude
Conscious/explicit attitudes not correlated with implicit attitudes
* Suggests that Ps not aware of implicit attitudes
Implicit but not explicit attitudes associated with change in satisfaction over
time (4 years)
Pros
* Avoid social desirability & reactivity problems
– Could be particularly useful for sensitive topics
Cons
* Big gap between construct of interest and operationalization
– Can we be sure that we are studying what we think we are studying?
DIFFERENT TYPES OF MEASURES: PHYSIOLOGICAL MEASURES
- Physiological responses = body’s reaction to various experiences/stimuli
– Autonomic nervous system activity (e.g., heart rate,
blood pressure)
– Hormone changes (e.g., cortisol, sex hormones)
– Immune system changes
– Brain activity
CHALLENGES COLLECTING &
INTERPRETING FMRI DATA
* Very confined & noisy environment
– Makes it challenging to create powerful
psychological experience for participants
* Don’t see brain “activation” per se—infer
activation by subtracting response on control trial from trial of interest
– Need to think carefully about task design
Pros
* Interesting in their own right (e.g., understanding link between relationships
and health)
* Outside participants’ control (not susceptible to social desirability bias, etc.)
Cons
* Very expensive à smaller sample size
* Ambiguity in interpretation
* Could be more invasive (depending on the measure)
DIFFERENT TYPES OF MEASURES: ARCHIVAL DATA
- Might be able to draw on publicly available documents & data
– E.g., marriage licenses, yearbooks, Facebook, personal ads - Data collected by someone else, often for an unrelated purpose
- E.g., more positive facial expressions in
yearbook photos predict likelihood of being
happily married 30 years later (Harker & Keltner,
2001)
Pros
* Typically economical
* Can examine historical trends
Cons
* Limited by type and quality of original data
MEASUREMENT:
TAKEAWAY
No single approach is perfect & free of
limitations
* Ideally, want to adopt a multimethod
approach—using combination of methods
to triangulate on an answer
STUDY DESIGN: CORRELATIONAL DESIGN
- Before: how will we operationalize construct of interest?
- Now: how will we collect our data?
- Multiple options
– Pros and cons
– Some may be appropriate for certain research questions
– Again, will often want to use a multimethod approach
CORRELATIONAL DESIGN
* Examine naturally occurring associations between variables (the things we
are measuring)
– E.g., do people tend to be attracted to those more similar to themselves?
* Strength of association captured by the correlation coefficient (r), which can
range from -1 to +1
– Sign tells us direction
– Magnitude tells us strength of association
Pros
* Sometimes the only option available
– Some variables researchers cannot manipulate—gender, culture, age,
marriage status, chronic illness, having an affair, etc.
Cons
* Can’t draw conclusions about causation (conclusion about cause and effect
EXAMPLE:
DOES MARRIAGE CAUSE HAPPINESS?
* Evidence to suggest that marriage & happiness are associated
* Plausible that marriage may increase happiness
* But could also be true that happier people are more likely to get married
* OR: optimistic people may be more likely to get married AND more likely to
be happy
CORRELATION IS NOT CAUSATION
* Three criteria must be met to
conclude causation:
1. Two variables must be correlated
2. One variable must precede the
other
3. There must be no reasonable
alternative explanations for the
pattern of correlation
CROSS-SECTIONAL VS.
LONGITUDINAL RESEARCH
* Cross-sectional data = data collected at one single point in time
* Longitudinal data = data collected from the same participants on multiple
occasions
– Allows us to examine change over time
- E.g., how does satisfaction change over the course of a relationship?
- Do communication patterns early in the relationship predict whether the
couple will stay together or break up later on?
STUDY DESIGN: STATISTICALLY CONTROLLING
STATISTICALLY “CONTROLLING” FOR
ALTERNATIVE EXPLANATIONS
* You measure an alternative variable that might explain your effect, and you
include it in your analyses
* If your effect is just a proxy for something the control variable is doing, your
effect will be wiped out
* E.g., When children eat more ice cream, they’re more likely to drown!
– Have you tried controlling for whether it’s summertime?
– Oh hey, the effect is gone. Wild.
* Although helpful, this type of statistical control is still subject to limitations
– Also, can be hard to anticipate every relevant variable
STUDY DESIGN: LONGITUDINAL RESEARCH
LONGITUDINAL RESEARCH:
CHALLENGES & CONSIDERATIONS
* Choosing the right interval
– For certain questions, may need to follow couples for a long time (months,
years, decades)
– Sometimes, may be interested in more frequent assessments over a shorter
period of time
* Daily diary study = type of longitudinal approach where Ps provide data
every day at about the same time
* Experience sampling = type of longitudinal approach where data is
gathered throughout the day, thereby capturing behaviours, thoughts, &
feelings as they occur
* The sample you start with may not be the sample you end up with
– Attrition bias = the participants who drop out may systematically differ
from those remaining in the study
* Example: researchers have often observed U-shaped pattern of marriage
satisfaction
– Could be an artifact of unsatisfied couples dropping out
LONGITUDINAL RESEARCH:
PROS & CONS
Pros
* Captures change over time
* Can examine processes that would be impossible/unethical to cause
– E.g., can’t “assign” people to get married, but can look at how marriage
affects them over time
* Daily diary & experience sampling: less subject to retrospective bias
– Capture real experiences as they happen
Cons
* Expensive, time- and labour-intensive
* Attrition bias
* Getting 1 step closer towards making causal claims, but still not there
CHOOSING PARTICIPANTS
- Population = all the people we are interested in
- Sample = a subset of that population
– Must be chosen carefully to ensure it represents the population
CONVENIENCE SAMPLES
* Anyone who is readily available
– E.g., undergraduate students
* Easier to get, BUT
* May not be representative of people in broader population
WEIRD PARTICIPANTS
* Because convenience samples are much easier and cheaper to get, they are
used more frequently
* Most studies have been conducted on people who are WEIRD
* WEIRD = Western
Educated
Industrialized
Rich
Democratic countries
REPRESENTATIVE SAMPLE
* A sample that resembles the entire population we want to study on the
variables of interest
– E.g., all nationalities, SES backgrounds
* E.g., national statistical agencies (like Statistics Canada)
* Very difficult to get
* Even if we do a good job of reaching out to representative sample, the people
who agree to participate may differ from those who don’t (volunteer bias)
KARNEY ET AL., 1995
* Mailed invitation to participate in longitudinal study to couples who had
obtained their marriage license in LA county between 1993-1994
* Those who responded to invitation:
– Higher SES (more years of education, higher status job)
– More likely to have cohabitated prior to marriage
TAKEAWAY
* Relationships researchers generally use a combination of convenience and
more (though not totally) representative samples
* Having a non-representative sample does not make the study wrong—it just
limits the extent to which we can generalize our findings
* Some relationship phenomena more universal than others
* Overarching theme: no one study is perfect
– Relationship science is an incremental proces
ETHICAL ISSUES
- Asked to deeply think about and confide about issues of a highly personal and
sensitive nature - May experience negative effects, like recognizing problems in relationship for
the first time (Bradbury, 1994) - Cost-benefit analysis:
– Need to consider not only the cost of doing studies, but also the benefit of
doing them (or the cost of not doing them) - Need to sensitive in how we approach our participants, provide effective
debriefing, counselling resources
STUDY DESIGN: EXPERIMENTAL DESIGN
- Manipulate one variable to determine effect on another variable
– The only design that (if properly executed) allows us to make causal claims - Independent variable = the manipulated variable in an experiment (possible
cause) - Dependent variable = the measured variable in an experiment (possible
effect)
IMPORTANCE OF A GOOD CONTROL
* Compare experimental group to control
group to determine effect of experimental
intervention
* Want the control condition to match
experimental condition as closely as
possible minus the key ingredient
– Want to avoid confounds = alternate
explanations for relationship between two
variables
EXAMPLE
* Are attractive people seen more positively?
* Have participants rate target profiles on positive & negative personality traits
– Manipulate attractiveness of photo: attractive vs. unattractive
* Must make sure photos match on all other factors—e.g.:
– Gender
– Age
– Race
– Quality/colour of photo
IMPORTANCE OF RANDOM
ASSIGNMENT
* Every participant in your study brings unique set of perspectives, biases, etc.
– Potential threat to internal validity
* Random assignment = every participant has equal chance of being assigned
to experimental or control group
– With large enough sample, our two groups should be similar on all these
individual traits
– This helps rule out alternative explanations
EXAMPLE
* Study to access effectiveness of couples therapy
* Two conditions: weekly therapy sessions for 6 months VS. no therapy (control)
* Allow couples to self-select into condition
* After 6 months, conclude that intervention worked
* Problem?
INTERNAL VALIDITY
50
Three criteria must be met to conclude causation:
1. Two variables must be correlated
2. One variable must precede the other
3. There must be no reasonable alternative explanations for the
pattern of association
Internal validity = can we rule out alternate explanations in the
experiment?
– Relies on selection of appropriate control & random assignment
LONGITUDINAL-EXPERIMENTAL
STUDIES
* Could expose couples to an intervention and track them over time
(longitudinal assessments)
* Same concerns about attrition bias apply
* E.g., recruit couples to test effectives of couples’ therapy
– 50% assigned to weekly therapy, 50% control
– After 6 months, experimental more committed to their marriage than
control group—yay!
– Not so fast! Turns out that large number of couples dropped out from
experimental group
* Weekly therapy is demanding
* Couples who remain in therapy may
have been more committed to begin
with
* So, despite use of random
assignment in the beginning, our
experimental and control groups are
no longer matched due to
differential attrition
EXPERIMENTS
Pros
- Allows us to make causal claims
- Caveat: as long as there are no threats to internal validity
Cons
* May have lower external validity = extent to which results obtained in a given
generalize to other contexts
* Not always an option
NATURE OF DYADIC DATA
- In order to examine dyadic processes, often want to get data from both
members of couple - Requires different analytic approach—need to account for dependency in the
data
– Fundamental assumption of regression: independence of the data
– Using regular approach may lead to bias in our significance tests
– Also, fundamentally interested in the interdependent nature of dyadic
processes
ACTOR-PARTNER INTERDEPENDENCE
MODEL (KASHY & KENNY, 1999)
Allows us to examine how individual outcomes are affected by BOTH one’s own
characteristics (actor effect) AND the partner’s characteristics (partner effect)
SEE IMAGE
DYADIC SAMPLES
* Often want to get dyadic data to examine dyadic processes
* But are dyadic samples representative of the population or unique?
* Individuals whose partner agreed to participate with them reported:
– Greater relationship satisfaction, commitment, more secure attachment
(Barton et al., 2020)
* Couples who agreed to participate together less likely to experience breakup
over time (Park, Impett, & MacDonald, 2020)
