Simpson's Paradox - Lecture 28 Flashcards
What is Simpson’s Paradox (SP)?
An observed correlation within a population could be reversed if applied to specific subgroups or individuals of that population
(e.g. a treatment which is effective at a populatino level might have adverse consequences within each population’s subgroup)
Scientific definition: A counterintuitive feature of data, which arises when inferences are drawn across different explanatory levels (e.g. population-subgroups, subgroups-individuals)
What is SP linked to?
CAUSAL INFERENCE:
!!! MANY TIMES WE MAKE THE FALLACY OF INTERPRETING AN OBSERVED ASSOCIATION AS A CAUSAL ASSOCIATION IMMEDIATELY !!!
What is some general information on SP?
- Occurs more frequently than we think
- There is inadequate attention to SP, which results in both incorrect inferences that compromize both the quest for truth and public health and safety
Example showcasing SP
Admission statistics for males and females
in two faculties (A and B) that together constitute the Berkeley
graduate program. Overall, proportionally fewer females than males were admitted into graduate school (84% males vs. 78% females). However, when the admission proportions are inspected for the individual faculties A and B, the reverse pattern is true: In both
school A and B theproportion of females admitted is greater than
that of males (97 vs. 91% in faculty A, and 33 vs. 20% in faculty B)
Other examples showcasing SP?
(This is not important to know, if you understood the above example, you’ll understand these as well. I’ll just mention them since there were a few, and if you want more explanations just come to me and I’ll explain them quicker that way)
- Alcohol - IQ
- Coffee - Neurotic
- Speed-accuracy tradeoff
The phenomenon observed in the above example is also called sign reversal. Why is sign reversal important?
e.g. assume we’re studying the effect of a drug on treating health problems
- Positive effect of the drug: Leads us to research it and invest in it more
- Negative effect of the drug: Don’t research it further, stop using it immediately
In effect sizes, although d1 = 0.5 and d2 = 0.9 have a bigger difference than d3 = - 0.15 and d4 = 0.15, the difference between d3 and d4 shows a more critical difference.
SP in individual differences
SP in individual differences
What is a wrong inference many people make regarding inter-individual differences?
Assume we’re studying personality: patterns of inter-individual differences are often thouht to be informative about psychological constructs. Many believe that personality dimensions play a causal role on individual’s behavior (e.g. extraversion causes party-going)
-> WRONG INFERENCE
Since we’re studying patterns of inter-individual differences, our findings are on the group level. Group-level findings can only genalize to the individuals when the data entail all possible values of a dimension and are very thorough, which is never the case. Even if we find that extraversion leads to party-going, we might find one person high on introversion that likes to go to parties, or another person high on extraversion that might in that moment not like to go to parties, or doesn’t go in general.
Controlling/Minimizing SP
Controlling/Minimizing SP
What is important to know regarding how to control SP?
There isn’t a single correct way for analyzing data to prevent SP
(One method which has been considered is conditioning on subgroups. A problem with this thoug is that it increases supriosu dependencies)
Controlling/Minimizing SP
What is the biggest danger that leads to SP’s?
Inferring that a finding on a group level generalizes to subgroups or individuals as well (Links to flashcard 8)
Controlling/Minimizing SP
What are some ways to prevent SP?
- Develop and test mechanisitc explanations
- Study change in individuals over time
- Intervene
Controlling/Minimizing SP
Develop and test mechanisitc explanations
Without well-developed top-down schemas we have a cognitive blind post in which we’re vulnerable in making wrong inferences: Blind spot leads to SP
SOLUTION:
1. Propose a mechanism and determine at which level its presumed to operate between groups, within groups, or within people)
2. Carefully assess whether the explanatory level at which the data were collected align with the explanatory level of the proposed mechanism
Controlling/Minimizing SP
Study change in individuals over time
(Says it in the title)
- Modern technology can help us study this change more effectively and easier
Controlling/Minimizing SP
Intervene
When we intervene, that ensures that teh relationship between two variables at the group level reflects a causal pattern in individuals over time (use an experimental study basically)
