CHAPTER 9 Why Correlation Doesn’t Imply Causation Flashcards
What does correlation not necessarily imply?
Causation
Correlation indicates a relationship between variables but does not confirm one causes the other.
What are the two key reasons why an observed correlation might be a biased estimate of a causal relationship?
- Confounders
- Reverse causation
Confounders are external factors that can influence both variables, while reverse causation occurs when the outcome affects the predictor.
Why is understanding the distinction between correlation and causation important?
It prevents making decisions based on misguided beliefs about how actions will affect outcomes
Misinterpreting correlation as causation can lead to significant errors in decision-making.
What is the definition of causation?
A change in some feature of the world that would result from a change to some other feature of the world
Causation implies that one event is the result of the occurrence of another event.
What is the main argument behind charter schools?
To encourage innovation and choice in education
Charter schools operate independently of the public school system and aim to provide alternatives to traditional education.
What happens when charter schools are oversubscribed?
Students are admitted by random lottery
This process ensures a fair chance for all applicants regardless of background.
What correlation exists between charter school attendance and academic performance?
Low-income students in charter schools have better educational outcomes than those in traditional public schools
This correlation is based on observed performance metrics such as test scores and graduation rates.
What is the significance of the Preuss School in the discussion of charter schools?
It has demonstrated outstanding academic outcomes for its students
The school serves low-income students and has a high rate of college admissions.
What critical question arises when evaluating the effectiveness of charter schools?
Does attending a charter school improve a child’s educational outcomes compared to attending a local public school?
This question addresses the core of the debate regarding charter schools’ impact.
How can we better assess the causal impact of charter schools?
By comparing the performance of lottery winners to lottery losers
This method accounts for the characteristics of students who applied to charter schools.
What was found when comparing lottery winners and losers at the Preuss School?
The correlation between charter school attendance and performance disappeared; no performance difference was found
This suggests that prior academic differences, not charter school attendance, may account for performance.
What conclusion can be drawn from studies on charter school effectiveness?
In many cases, there is no evidence that winning a charter school lottery impacts achievement
This highlights the importance of comparing similar groups to draw causal inferences.
What are potential outcomes in the context of causal relationships?
They represent the hypothetical outcomes for individuals based on whether they received a treatment or not
This framework helps clarify the causal impact of interventions.
What is the treatment variable represented by T in causal analysis?
Going to the charter school
T = 1 indicates attendance at a charter school, while T = 0 indicates attendance at a public school.
What does T = 1 indicate for an individual in the context of school attendance?
T = 1 indicates that the individual attended the charter school.
What does T = 0 indicate for an individual in the context of school attendance?
T = 0 indicates that the individual attended a public school.
What are the two potential standardized test scores for an individual based on school attendance?
Y1i (if attended charter school) and Y0i (if attended public school).
What is the effect of going to the charter school on person i’s test scores denoted as?
Y1i - Y0i.
What is the Average Treatment Effect (ATE)?
ATE is the average effect of going to a charter school across a population.
What is the population difference in means?
The difference in average test scores between charter school students and public school students.
What is the Average Treatment Effect on the Treated (ATT)?
ATT is the average effect of going to the charter school among those who attended the charter school.
What is the Average Treatment Effect on the Untreated (ATU)?
ATU is the average effect of going to the charter school among those who attended public schools.
True or False: The ATE can be directly observed.
False.
What does the bias term indicate when estimating the ATT?
It indicates that the two groups have baseline differences.
What are baseline differences?
Differences between treated and untreated groups that exist even in a counterfactual world.
What does the correlation between test scores and charter school attendance potentially indicate?
It may indicate a bias rather than a causal relationship.
What is the role of randomization in estimating causal effects?
Randomization helps ensure that treated and untreated groups are statistically similar.
What are confounders?
Features of the world that affect both treatment status and outcomes.
What are the two conditions for a variable to be considered a confounder?
- It affects treatment status. * It affects the outcome independently of treatment status.
What example is given to illustrate the concept of confounders?
Academic talent as a confounder affecting both treatment and outcomes.
What is reverse causality?
When the outcome affects treatment status.
What example is provided to illustrate reverse causality?
The effect of civil war on a country’s economic prosperity.
What should be assessed to determine if a correlation is causal?
The presence of confounders and potential reverse causality.
How does reverse causality create baseline differences?
It leads to systematic differences in outcomes that are not due to treatment effects.
What does the ATE represent in relation to ATT and ATU?
ATE is a weighted average of ATT and ATU.
What is reverse causality?
Reverse causality is when the outcome affects treatment status
It raises concerns about interpreting correlations as causal relationships.
What should you check for before interpreting a correlation as causal?
Potential sources of reverse causality
Consider if the causal arrow might run from the outcome to the treatment.
What are the two key sources of bias for estimating causal relationships mentioned?
- Confounders
- Reverse causality
These sources can distort the true relationship between treatment and outcome.
What is the 10,000-Hour Rule?
The idea that great success is achieved by committing to 10,000 hours of deliberate practice
Popularized by Malcolm Gladwell, it suggests that talent is secondary to practice.
Who inspired Dan McLaughlin’s pursuit of professional golf?
K. Anders Ericsson
Ericsson’s research focused on the importance of deliberate practice for high performance.
What distinguishes true masters from good experts according to Ericsson?
The amount of time devoted to deliberate practice
Deliberate practice is specifically targeted to improve performance.
What was the finding of Ericsson’s study on violinists?
The best violinists practiced at least 10,000 hours by age eighteen
The least accomplished practiced about 5,000 hours.
What is a confounder?
A variable that affects both the treatment and the outcome
It can distort the apparent relationship between them.
True or False: Innate talent is not a factor in achieving high performance.
False
Innate talent can influence the amount of deliberate practice and success.
What does the correlation between practice and success not necessarily imply?
A causal relationship
Other factors, like innate talent, can confound this correlation.
What is the consensus among nutrition experts regarding diet soda?
Diet soda is considered bad for health
Linked to obesity, diabetes, and heart attacks.
Fill in the blank: There is a negative correlation between drinking diet soda and _______.
health outcomes
People who drink diet soda are more likely to experience health issues.
What are two potential confounders in the diet soda and health outcomes correlation?
- Snacking
- Obesity/Diabetes
These factors could influence both soda consumption and health status.
How can reverse causality be conceptualized in terms of confounders?
Anticipated outcomes can act as confounders
For example, beliefs about civil war risk can affect economic conditions.
What is a key concern when interpreting the correlation between practice and performance?
The role of innate talent as a confounder
Talent may bias the interpretation of the impact of practice on success.
What is the effect of anticipating a future civil war on a country’s economy?
It can deter investment, lead to capital flight, and cause brain drain
This creates a weaker economy due to perceived civil war risks.
What is reverse causality in the context of civil war risk and economic weakness?
It suggests that civil war risk causes economic weakness.
Define confounding in relation to civil war risk.
Confounders are factors that lead to the belief that a country is at high risk of civil war, causing economic weakness.
What is the primary reason political candidates spend time raising money for campaigns?
They believe campaign dollars are essential for their electoral prospects.
Who conducted an influential study on campaign spending in 1978?
Gary Jacobson.
What did Jacobson conclude about campaign spending for challengers and incumbents?
Spending helps challengers but has little benefit for incumbents.
How is challenger spending correlated with their electoral success?
It is strongly positively correlated with challengers’ vote shares.
What correlation does incumbent spending have with their electoral success?
It is negatively correlated with incumbents’ vote shares.
What might too much spending by an already well-known incumbent do?
It might annoy and turn off potential supporters.
What is a concern regarding electoral strength in interpreting campaign spending data?
Electoral strength may be a confounder affecting both fundraising and electoral success.
True or False: Incumbents typically spend more money when they are electorally strong.
False.
What do subsequent studies suggest about campaign spending effects?
They generally suggest positive effects for both challengers and incumbents.
What is often required to meaningfully influence the outcome of a large election through campaign donations?
Tens of millions of dollars.
What does signing the bias refer to in the context of correlation and causality?
It assesses whether the correlation over- or under-estimates the causal effect.
If the bias in an estimate is positive, what does this imply about the observed correlation?
It is an over-estimate of the true causal effect.
If the bias in an estimate is negative, what does this imply about the observed correlation?
It is an under-estimate of the true causal effect.
What is a confounder in the context of charter school attendance and standardized test scores?
Students applying to charter schools may be more academically gifted.
Does the confounder of academic talent lead to an over- or under-estimate of the effect of charter school attendance on test scores?
Over-estimate.
What happens when a confounder has opposite signed effects on treatment and outcome?
It creates negative bias, leading to an under-estimate of the true effect.
What is the relationship between living in a poor neighborhood and charter school attendance?
It has a positive effect on the likelihood of attending a charter school.
What does the observed correlation between charter school attendance and test scores reflect when confounders are present?
It reflects the influence of confounders, leading to bias in the correlation.
How can one estimate the extent of bias if data on confounders is not available?
By making guesses about the confounder’s effect on the outcome and its correlation with the treatment.
What is the implication of assuming the true effect is zero in evaluating bias?
It helps to determine how large the bias would have to be to explain the observed correlation.
What is the main reason simple correlations are not preferred for learning about causal relationships?
Bias in the data
Simple correlations can lead to misleading conclusions about causality.
What is sensitivity analysis?
An approach that starts with the assumption that the true effect is zero and assesses how large the bias would need to be to explain an observed correlation
This method helps evaluate the plausibility of observed correlations.
What is a confounder?
A variable that affects both the treatment and the outcome
Confounders can distort the perceived relationship between the treatment and the outcome.
True or False: Correlation necessarily implies causation.
False
Correlation does not imply causation due to potential confounding factors.
What did the 2012 study in The Lancet Infectious Diseases investigate?
The relationship between hormonal contraception use and HIV transmission
The study analyzed data from couples with one HIV-infected partner.
What were the two significant findings of the Lancet study regarding hormonal contraception and HIV?
- HIV-negative women using hormonal contraception were twice as likely to acquire HIV
- HIV-infected women using hormonal contraception were twice as likely to transmit HIV
These findings were reported widely but raised concerns about confounding variables.
What is a mechanism in the context of causal relationships?
A feature of the world that the treatment affects, which then affects the outcome
Mechanisms explain how a treatment produces its effects.
How can sexual activity act as a confounder in the study of hormonal contraception and HIV transmission?
More sexually active women are more likely to use hormonal contraception and also at greater risk of HIV transmission
This can lead to an over-estimation of the causal relationship.
What is the distinction between confounders and mechanisms?
Confounders are pre-treatment covariates affecting both treatment and outcome, while mechanisms are post-treatment covariates affected by treatment and then affecting the outcome
Understanding this distinction is crucial for causal inference.
In the context of a study on statins, what could be a confounder?
Wealth
Wealth can affect both the likelihood of taking statins and the risk of dying from heart disease.
What could be considered a mechanism in the statin study?
Lower cholesterol
If cholesterol levels are measured after starting statins, then it is a mechanism of the treatment’s effect.
True or False: Democracy can be both a confounder and a mechanism in studies of civil war risk and economic factors.
True
Democracy can influence both economic conditions and civil war risk, making its role context-dependent.
What should you assess when someone presents a correlation as evidence of a causal relationship?
- Are we actually observing a correlation?
- Does the estimated correlation reflect a genuine relationship?
- Is this correlation convincing evidence of a causal relationship?
These questions help critically evaluate the validity of the claimed relationship.
What is noise in the context of causal inference?
Idiosyncratic factors that affect the estimate, separate from any causal relationship
Noise can arise from sampling variation or other unrelated variations.
What are the two kinds of ways an estimated correlation can deviate from the causal effect of interest?
- Noise 2. Bias
Noise refers to idiosyncratic factors affecting the estimate, while bias involves confounders or reverse causation.
Define ‘noise’ in the context of correlation estimates.
Idiosyncratic factors that affect the estimate, which can arise from sampling variation or measurement errors.
What does ‘bias’ refer to when discussing correlation and causation?
Confounders or reverse causation that makes the estimate different from the estimand on average.
What is a ‘spurious correlation’?
A correlation that occurs between two variables without a logical or causal connection.
What correlation coefficient (r) indicates a very strong correlation?
0.992
What example illustrates a correlation between suicides by hanging and government spending on science?
The correlation over time showing higher hanging suicides coinciding with higher science spending.
What might explain the correlation between hanging suicides and science spending?
Potential confounders like population growth affecting both variables.
What does the Average Treatment Effect (ATE) measure?
The difference in average outcome comparing treated and untreated populations.
Fill in the blank: A confounder is a feature of the world that _______.
[1] has an effect on treatment status and [2] has an effect on the potential outcome.
What is the Average Treatment Effect on the Treated (ATT)?
The difference in average outcome comparing the treated subgroup to the counterfactual scenario where they are untreated.
What is the Average Treatment Effect on the Untreated (ATU)?
The difference in average outcome comparing the untreated subgroup to the counterfactual scenario where they are treated.
What is an example of a correlation attributed to noise rather than causation?
The correlation between Nicolas Cage movies and swimming pool drownings.
Identify two potential confounders that might affect the correlation between marriage and happiness.
- Socioeconomic status 2. Pre-existing mental health conditions
Both confounders could influence both likelihood of marriage and levels of happiness.
What does it mean if an estimate is an over-estimate?
The bias is positive, making the estimate larger than the true effect in expectation.
What is the difference in means?
The difference in average outcome comparing treated and untreated subgroups.
How can data analytics improve university fundraising according to the example provided?
By identifying correlations that suggest strategies to encourage alumni giving.
What conclusion can be drawn from the correlation between sociology doctorates and space launches?
It is likely due to noise rather than a causal relationship.
What is the effect of population growth on the correlation of hanging suicides and science spending?
It could plausibly increase both suicides and science spending.
What does the term ‘baseline differences’ refer to?
Differences in average potential outcomes between two groups with the same treatment status.
What is the implication of a correlation not holding outside the sample data?
It suggests the correlation may be a result of noise.
What happens to the life satisfaction of people who become divorced according to the study mentioned?
It decreases significantly after divorce.
What challenge does the correlation between Nicolas Cage movies and swimming pool drownings pose?
Conceptualizing the correlation as noise despite having complete data.
What does the concept of a ‘mechanism’ or ‘mediator’ refer to?
A feature of the world affected by treatment that then affects the outcome.
What change in happiness is studied in relation to the year spouses passed away?
The change in happiness before, during, and after the year in which their spouses passed away.
Does the study of happiness before and after the death of a spouse make you more or less confident in Gilbert’s causal interpretation?
This question prompts personal reflection on confidence in Gilbert’s interpretation.
What dataset is suggested for downloading to analyze House Elections Spending in 2018?
HouseElectionsSpending2018.csv and README.txt.
What relationship is analyzed using linear regression in the House Elections Spending data?
The relationship between incumbent vote share and incumbent spending.
Is the correlation between incumbent spending and vote share positive or negative?
This requires analysis of the data to determine.
According to the data, do incumbents who spend more do better or worse?
This requires analysis of the data to determine.
What should be interpreted regarding the magnitude and direction of the correlation between incumbent spending and vote share?
This requires analysis of the data to determine.
What should be done similarly for challengers in the House Elections Spending analysis?
Run a linear regression to find the relationship between challenger vote share and spending.
What are three confounders you should identify when assessing evidence of campaign spending effects?
This requires personal reflection to identify potential confounders.
Do you have any variables in the dataset that measure the confounders identified?
Identify a variable that might plausibly measure a confounder in the dataset.
Using linear regression, what should be assessed regarding spending and potential confounders in the dataset?
Assess whether incumbent spending and challenger spending are correlated with a potential confounder.
What is an example of a misinterpretation of correlation as causal relationship?
Provide a specific example from a researcher, journalist, or analyst.
What should be explained regarding the evidence presented in the misinterpretation example?
Explain why the correlation is not persuasive evidence of the purported causal relationship.
What should be discussed in terms of the likely direction of bias in the misinterpretation example?
Discuss the likely direction of the bias related to the misinterpretation.
What should be considered as a better way to estimate the causal relationship of interest in the example?
This requires personal reflection on better estimation methods.
Who authored the study of the Preuss School at UCSD?
Larry McClure, Betsy Strick, Rachel Jacob-Almeida, and Christopher Reichher.
What is the main focus of the study ‘Who Benefits from KIPP?’
Analyzing the effects of the Knowledge is Power Program.
What is the focus of the study by Betts et al. regarding charter schools?
The effects of school choice on student integration and achievement.
What is the significance of the study by Ericsson et al. in relation to expert performance?
It discusses the role of deliberate practice in the acquisition of expert performance.
What does the study by Heffron et al. investigate?
The use of hormonal contraceptives and risk of HIV-1 transmission.
What does Jacobson’s study from 1978 analyze?
The effects of campaign spending in congressional elections.
What book discusses the concept of spurious correlations?
Spurious Correlations: Correlation Does Not Equal Causation by Tyler Vigen.
What is the main finding of the study by Zimmermann and Easterlin on happiness?
Examines happiness in relation to cohabitation, marriage, divorce, and happiness.
What do Gardner and Oswald’s findings suggest about divorcing couples?
Investigates whether divorcing couples become happier by breaking up.
What is a weighted average?
An average where different weights are assigned to different items.
