chapter 12 Flashcards
What is Generalizability? … and why do we even care?
- Extent you can generalize beyond your limited study situation
- It is important because research informs treatments, interventions and further research studies.
- Treatments & interventions for 1 group may not apply to all
- It’s the ability to extend findings beyond the original study context.
- Is it relevant? Does it matter, would the results occur with different people in different locations?
Generalizability important for applied psychology.
Research should be used to improve lives.
- Goal of psychology is to promote human welfare.
- Impact of psych research:
- health: helped shaped mental health treatments
- Law and criminal justice
- Education
- Work environments
- In order for any research to be helpful, findings must be replicable and generalisable.
Challenges To Generalizability
Multiple factors influence generalizability
Selection: Participation, typically not selected from the general population
- be mindful that the group you’re studding could be applied to broader pop
History & Sociopolitical climate
- events that occur during the time period of the study impact the study.
Experimental Setting
- Mundane () vs experimental realism (evoking emotions)
QUESTION on exam about this
Participant Selection & Generalizability
University Students-can researcher done on university students be applicable (it’s a convince sample)
- Knowledge of psychology
- Similar age groups: early adulthood
- High levels of intelligence
Criticism
- social political values in a flux
- higher need for peer approval
- higher levels of intelligence
- WEIRD
Participants selection and generalizability
WEIRD areas
96% of participants
w- western
e- educated
i- industrialized
r- rich
d- Democratic
ON exam- it’s a criticism of using university students
- in order for replication to happen must involve another group
Undergraduate students are weird, so why are we continuing to use them?
- allows us to gain insight good start
Limits of Convenience Samples
Why do we continue to use students?
- Students are increasingly diverse and representative of society
- Important to question assumptions
- Replicability
- By better understanding what happens beyond university students, results are more generalisable
Participant Selection & Generalizability
Volunteers issues.
- Do the results differ from people who would not volunteer?
- More highly educated
- high levels of conscientiousness
- Individual differences among volunteers (diff kinds of people vary between experiments)
- Motivation - how you’re recruiting
- Different people will come depending on what you’re offering, especially if you’re offering nothing.
Participant Selection & Generalizability
Gender Considerations Issues
- Binary choice is an issue (there are many genders)
- Mixed or single genders
- gender biases
- Is interpretation consistent across all genders?
- We use gender as an independent variable within analysis
Cultural Considerations
- Can you extend beyond Racial and ethnic composition
- Used to be predominantly white male
- Big push to generalize outside of sample
- Operational definitions can be influenced by culture
The Role of the Experimenter
Experimenter’s influence must remain constant
Personality characteristics and gender
- Participants responding to a warm vs cold researcher.
Multiple experimenters and being consistent minimizes bais
Whether the results can extend to other researchers
Solution to generalization
Direct Replication
Direct Replication
Direct replication, do the exact same study over.
- if it succeeds, strengthens findings
- if it fails, find out why
- Makes psychological science more trustworthy
Copy the procedure precisely
* Ascertain if same results
obtained
* Failures due to flaws in original, replication procedure or random error
Direct Replication & Disciplinary Reforms:
* Trying to figure out what’s the most helpful for the field.
* Becoming easier to publish direct replications
* Many labs projects
Conceptual Replication
*Test same hypothesis but uses new measures or methods to do so
ex. Journaling and expressing gratitude maybe make it talking
Helps us extend the understanding, but it doesn’t replace.
The use of different procedures
to replicate a research finding
* Can manipulate IV differently
* Can measure DV differently
* Do not replace direct
replications
* Can manipulate IV differently
* Can measure DV differently
* Do not replace direct
replications
Literature Review
*A tool used to evaluate replications
- used to identify trends areas of research that has gaps
*Summarizes what has been found
* Exposes inconsistent findings
and areas lacking proper research
* Tells what findings are strongly/weakly supported
* Discusses future directions
*provides big picture
*gaps
*ideas for new research
Meta-Analysis
- Take data from all tests.
*What does the data say on all these topics?
*if I convert them to similar structure, do the results hold?
Method for determining the
reliability of a finding by
examining results from
many studies - Pools actual results and
analyzes them statistically - Find average effect size in
literature
Go beyond literature reviews look at the data
Meta-Analysis & The File Drawer Problem
*
ex. Research on antidepressants almost all + results were published in medical journals
- 74 analyzed 38 + 12 - most of + published, most of - not.
- Creates a distorted view of treatments.
- Transparency in research reporting.
Null/opposite results rarely published
* Collect unpublished studies
* Strong effects minimize this
problem
