Class 22 Flashcards
Do some people work on methods, and others work on theories?
Often go hand in hand
IAT and Person who developed IAT
- The best methods actually challenge /extend / refute theories
- Push for new methods allows for more theories
When applied effectively and thoughtfully, new methodological approaches have the potential to resolve ongoing theoretical
debates and open new areas of research that were previously impossible due to methodological limitations
Why do we still have debates over theories in the literature?
Because the methods that we have give us two equally plausible interpretations of this effect
(can’t make progress on debate till new methods)
Who gets Nobel prizes?
People who identify a method
And who can show that the method shows things we didn’t know previously
(methods matter!)
A good method can open up theory
Explain Method 1: Social Network Analysis
Social network analysis (SNA) is the process of investigating social structures through the use of networks. It characterizes networked structures or individuals in terms of nodes (individual actors) and the ties, edges, or links (relationships) that connect them
Social Network Analysis
What is Nodes
What is ties, edges, or links
nodes: (individual actors)
- Entity in a network (dot)
ties, edges, or links: (relationships)
Social Network Analysis
What is ‘Directed Edge’?
Edge that has an orientation (e.g., an arrow indicating popularity)
- Ppl might list them as friends more then they list others
Social Network Analysis
‘Weighted Edge’
Indicates the strength of a
relationship (e.g., line thickness indicating how close two friends are)
Social Network Analysis
Distance
Smallest number of ‘edges’ needed to
connect two nodes
Social Network Analysis
Centrality
Importance of a node in the network
(e.g., how many edges a node contains or how many cross-group edges a node has)
Social Network Analysis
‘Community’:
Degree to which nodes in a network are connected to one another
Is it interconnected or cliquish
What are Advantages of social network?
- Can provide insight that people may not be able to self-report (e.g., who is actually most popular versus who is perceived to be
most popular). - Can show the spread of influence or change in a network. (can map change)
- Can identify popular nodes to target for possible interventions.
What are some limitations of social network analysis?
- Analyses will only be as good as how much of the network you cover. (you gotta get every middle schooler to map full relationship)
- People belong to multiple networks simultaneously (work, family, school,
clubs, sports, etc.) so effects in one network may or may not carry over
Explain how Paluck, Shepherd and Aronow (2016) used a social network approach to
design and assess the effectiveness of an anti-bullying intervention:
They identified ‘social referents’, meaning kids in a school network that had many connections with other kids, and they reasoned that these social referents would receive more attention from other kids and be looked to for information about group norms. (spread the peace message more easily)
Social networks were created by asking kids to list those other classmates who they had spent time with over the prior week
A random sample of students was selected in each school to receive the intervention, which required meeting with a research assistant every other week and discussing common conflict behaviors at school.
Throughout the intervention, these “seed students” were then encouraged to become the public face of opposition to these conflicts. (Discussion based: kids themselves came to a solution on what the problem was at their school - usually bullying)
(Notably, this intervention lacked an educational or persuasive unit regarding adult-defined problems at school)
Paluck, Shepherd and Aronow (2016) used a social network approach to
design and assess the effectiveness of an anti-bullying intervention:
What did this study find?
Schools in the treatment condition saw less conflict than those in the control condition
Less expulsions (meaning less bullying activity)
When was the Social Network Analysis study more effective?
The intervention was also more effective when a greater number of the
‘seed’ students in the intervention condition were ‘social referents’
Seed students: Kids with discussion intervention
Social referents: Kids who were central
Explain this Social Network Analysis:
Parkinson et al. (2017) used a social network analysis to see how people
encode social networks automatically?
An entire cohort of business school students were surveyed about their
social network (who they like to spend time with)
(business school students love networking)
Take these ppl and show them their cohort mates in an fMRI
- Manipulate:
- 1 social relationship from u (friends)
- 2 social relationship from u (friends of friends)
- etc
see if passively viewing diff relationships show up differently
What did the study on business students find?
Higher brain activation in 3 areas when they are closer to you
The current results indicate that when encountering familiar individuals, humans may spontaneously retrieve knowledge of where they are located, relative to oneself, in a mental map of ‘social space’
What does the business results suggest?
It’s not just race and gender we automatically pay attention to
- But it’s also complex social info we see is encoded quickly: “How are you related to me in my complex social network”
Explain Method 2: ‘Distributional Language Analysis’?
Archival analysis (a lot of text)
Looks into which words are most likely to occur with one another
The word “dog”, for example, is represented as more similar to “cat” than to “banana” because contexts containing “dog” are more similar to contexts containing “cat” than to contexts containing “banana”
What is the core assumption of Method 2: ‘Distributional Language Analysis’?
Words that co-occur with one another are more likely to have an association within that culture
(and likely have similar meanings)
What are some Advantages of distributional language analysis?
- Can provide insight that people may not be able to self-report
(e.g., what kinds of patterns exist in what we read).
(can’t self report pattens for every wiki article u read) - Allows for possible historical analyses for associations that may have existed before more modern measures.
- Easy to analyze change over time or differences between countries. (are implicit associations changing in a culture BEFORE we used IAT)
What are some limitations of of distributional language analysis?
- Requires a lot of data (tens of millions of words).
- May be dependent on the type of text you use. (maybe biases/ does it represent the culture)
- Cannot know whether the text is supporting (“men are better at work”),
reflecting (“the culture believes men are better at work”) or refuting (“it’s impossible that men are better at work”) certain association
Distributional Language Analysis
Across 25 languages. Lewis & Lupyan (2020) investigated the association between
1) The strength of the gender-career stereotype in a distributional language
analysis (e.g., how closely “man”/”career” and ”woman”/”home” co- occur vs. the opposite pairing).
2) The strength of the gender-career IAT effect among participants speaking
that language from Project Implicit
What did she find?
We find that the implicit (but not explicit) gender associations of participants in a country is correlated with the gender associations embedded in the dominant language spoken in that country
“Man = Career” societies have higher bias on the IAT
What did a follow up study on gender and Distributional Language Analysis find?
follow-up study suggests that these text-based gender stereotypes may
be even stronger in kids’ books
What is an important point on Distributional Language Analysis
(correlational analysis)
This is only a correlational analysis, so hard to determine causation.
To what extent does the language we make or see create implicit gender
stereotypes and to what extent do our implicit gender stereotypes create
changes in our language?
What did a ‘Distributional Language Analysis’ study on gender-career associations across
- different time periods (19th century vs. 21st century),
- audiences (children’s vs. adult books) or
- medium (books vs. spoken words) find?
Found gender-career associations across all three of them
How can u use ‘Distributional Language Analysis’ on one career?
can also be studied at the individual word or profession (for example,
how often the word ‘janitor’ co-occurs with ‘man’ vs. woman’).
These profession-level estimates of gender stereotypes from text are strongly correlated with actual gender disparities in employment
What is Method 3: ‘Big Data’?
archival analysis that uses a dataset too large to code by hand and instead relies on automated data collection and analysis (for example, webscraping of tweets / police reports)
some ‘big data’ approaches more recently are starting to also use experimental designs (for example, making twitter bots that respond to ‘fake news’ stories with either support or criticism
What are some Advantages of ‘big data’ approaches?
- Same strengths as archival research but just with greater scope.
- Often measures impactful, ‘real-world’ behavior, can have access to large
amounts of data that make any findings unlikely to be a “fluke
What are some limitations of ‘big data’ approaches?
- Basically, same limitations as archival research but now with less ability
to notice possible errors in data collection. (not sure if ur scraper is acting correctly) (hard to see if theres an error) - Analysis dependent on the data that are made available, often hard to get at issues of causality, potential threats to internal validity
What was a funny issue that shows a limitation of big data stuff?
Guy coded blogs after 9/11 to see if more negative / positive language was used
Found that ppl were using negative words
BUT: for an unrelated reasons, many of these sites started going offline
(so he was coding “error 404” “network connectivity issues” as negative, cuz error, or issues are negative)
Someone found this mistake and bro had to take back his paper
Something might contaminate ur data ur unaware of
Explain the ‘Big Data’ study on Police Records?
(3 year long legal battle to get records)
millions of timepoints related to police behavior in Chicago (merging daily
2.9 million patrol assignments with information like officers’ demographics, stops, arrests, and use of force).
(so when someone goes out to work beat and comes back = one data point)
FOUND=
- Black officers more likely to be assigned to racially diverse neighborhoods.
- Non-White officers engaged in less instances of force compared to
White officers.
- Female officers engaged in less instances of force compared to male officers
‘Big Data’ study on Police Records
What were these effects were particularly driven by?
Race of target (civilian)
these effects were particularly driven by interactions with Black civilians
SO:
Based on these models, deploying Black officers instead of White officers yields 12.55 fewer stops of Black civilians per 100 shifts, a reduction equal to 39% of typical
White-officer volume.
By contrast, Black officers make only 1.31 fewer stops of white civilians per 100 shifts than their White counterpart (so black officers aren’t stopping more white civilians)
Explain the ‘Big Data’ study on Medical Records?
(treatment algorithms)
worked with a large hospital to track the
treatment records of 50,000 patients over three years
Checked: enrollment in a “high-risk care management program”, which provide additional resources to patients with complex health needs
Enrollment in the program is primarily determined by a ”risk algorithm” based on
patient records, with higher scores leading to automatic enrollment in the program
(how sick do we think this person actually is)
Explain the ‘Big Data’ study on Medical Records?
(treatment algorithms)
FINDINGS:
when Black and White patients had the same risk score as determined by the algorithm, Black patients had worse objective health measures.
- For instance, for a Black and White patient to earn the same risk score to gain eligibility for the program, Black patients typically had 26% more chronic illnesses
Takeaway: White ppl did not need to be as sick to get into the program
Explain the ‘Big Data’ study on Medical Records?
(treatment algorithms)
We know the algorithm was not using race to decide (it was not coded)
So how could this be?
they found that it was using health costs (how much patients spend to maintain their health) to predict overall health
there is a difference between
“receiving health care” (health costs) and
“needing health care” (objective health).
Treating health costs as a proxy for health needs introduced a racial disparity; Black patients generate less healthcare costs than
White patients
What did the ‘Big Data’: Medical Records study reveal about bias?
This work represents a key problem identified by work on “algorithmic bias” – even systems designed to ignore race in an
effort to remove racial biases may still recreate such biases through reliance on proxy variables
Explain Method 4: Large Language Models?
No consistent framework for when to use
varies on how much u can use them and when
Large Language Models: Example 1 (article in scientistic paper)
Ran study among ppl who believe in conspiracies
(moon landing was faked)
Have them interact with AI chat bot
- About the conspiracy you believe in
Programmed to decrease ur belief in the conspiracy
After 3 rounds did reduce belief in the conspiracy by 40% (persisted for 2 months)
How might the Ai chat bot be applied to intergroup relations?
Discuss with chat bot ur views on some groups (maybe it’ll change ur mind)
Large Language Models: Example 2 (historical text)
Feed in all historical text about ancient greeks to run studies on (???)
Run psychological / behavioural studies on them (why would u do this)
