Reading 1 - a tale of two cultures (contrasting quantitative and qualitatie research) + data-set observations vs causal-process observations 2000 US presidential election

Question 1

A Tale of Two Cultures: Contrasting Quantitative and
Qualitative Research

Answer 1

James Mahoney + Gary Goertz

qualitative + quantitave research are distinct cultures marked by diff values, beliefs and norms
-> each is skeptical/suspicious of the other + communication across cultures tends to be difficult and marked by misunderstanding
-> both aim to produce valid descriptive and causal inferences

they are contrasted on:

1. approaches to explanation
2. conceptions of causation
3. multivariate explanations
4. equifinality
5. scope and causal generalization
6. case selection
7. weighting observations
8. substantively important cases
9. lack of fit
10. concepts and measurement

aim article = to contrast assumptions and practices of the two traditions to enhance cross-tradition communication
!focus on causal analysis

conclusion: labels quantitative and qualitative do a poor job capturing the real differences between the traditions (quantitative is not just about nrs, qualitative often uses nrs) -> other labels may be better: statistics vs logic, effect estimation vs outcome explanation, population-oriented vs case-oriented

Question 2

a tale of two cultures

1. approaches to explanation

Answer 2

qualitative = explanation of outcomes in individual cases within the scope of the theory under investigation

causes-of-effects approach: starts with cases and outcomes and moves backward toward the causes

• e.g. what caused WW1, the end of the cold war, rise of neopopulist regimes
• good theories must ideally explain the outcome in all the cases within the population (e.g. all cases of social revolution)
• similar to “normal/natural science”, e.g. what caused the space shuttle Challenger to explode?

quantitative
controlled experiment paradigm: you don’t know the outcome until treatment has been applied

follow the effects-of-causes approach: estimate the average effect of one or more causes across a population of cases (no interest in explaining specific outcomes in particular cases)

• yes: what is the effect of eco dev on democracy?
• no: was eco crisis necessary for democratization in the Southern Cone of Latin America?

complementarity: mixed-method research is possible

• explanation of an outcome in one or a small nr of cases -> question if same factors work in broader scope -> want to estimate average effect
• average effect -> wonder if it makes sense in a specific case

Question 3

a tale of two cultures

2. conceptions of causation

Answer 3

qualitative =

• causation in terms of necessary and/or sufficient causes
  necessary condition: without X Y would not have occurred so X causes Y
• INUS approach to causation (with small or medium N) = INUS cause is neither individually necessary nor individually sufficient for an outcome, it is one cause within a combination of causes that are jointly sufficient for an outcome

quantitative

• correlational approach to causation = a cause on average affects the values of an outcome across a large population
  realized causal effect: causal effect = (mean) y treatment - (mean) y control

-> kinds of hypotheses in the traditions are not always commensurate, sufficient causes cannot be unproblematically translated to language of correlational causation

scholars should be open to working with different conceptions of causation

Question 4

a tale of two cultures

3. multivariate explanations

Answer 4

qualitative: assumption that individual events do not have a cause, one must include a variety of causally relevant factors

quantitative = impossible to estimate average effects without controlling for relevant variables

Multivariate analysis in which (3) is of qualitative method and (4) is quantitative
(3) = Boolean model based on the INUS approach to causation (* represents AND, + represents OR, = represents sufficiency) -> two different combinatinos of variables are sufficent for the outcome
(4) standard statistical model that includes an interaction term

Differences:

- Qualitative = focus on impact of combinations of variables (rarely focus on effects of individual variables, no focus on net effect of one variable)
	○ It makes no sense to ask what is the effect of cause C because sometimes it has a positive effect, sometimes a negative effect depending on the other variable values
	○ It makes no sense to generalize about the overall effect of B without saying something about the context inw hich B appears
- Qualitative = focus on effect of individual causes (even when including an interaction term)
- Multiplication * in both models has diff meaning (logical AND vs multiplication)

Diff in language without recognition diff meaning -> quantitative scholars believe that a Boolean model is a set of interaction terms that could easily be translated into statistical language
It is better to understand the differences than to argue about which is better

Question 5

a tale of two cultures

4. equifinality

Answer 5

equifinality = multiple, conjunctrual causation = multiple causation

= idea that there are multiple causal paths to the same outcome

• strongly associated with qualitative comp approach by Ragin + in QRM seen in INUS approach
• discussions of equifinality are absent in quantitative work

!diff from interaction terms and statistical models bc there are only a few causal paths to a particular outcome

why this diff? quantitative research does not seek to explain any particular case

-> moving from a superset to particular subsets more stable qualitative findings more stable

Question 6

a tale of two cultures

5. scope and causal generalization

Answer 6

qualitative = narrow scopes -> inferences generalizable only to a limited range of cases

• conviction that causal heterogeneity is the norm for large populations: as population size increases, potential for key causal relationships to be missing/misspecified in theories increases (adding cases risks having to change the model)
  a causal path may not be sufficient for the outcome in the new cases

quantitative = broader scope + generalizations about large N

• goal to estimate average effect -> exclusion of certain variables relevant for a handful of cases not a problem, relegated to the error term

-> causal generalizations in qualitative work are more fragile than those in large N statistical analyses (esp when moving from subset to a superset)

Question 7

a tale of two cultures

6. case selection practices

Answer 7

qualitative = select cases where the outcome occurs (positive cases) + may also select negative cases to test theories

quantitative = selection without regard of DV value (bc that can introduce bias), random selection on IV

diff leads to debate across the traditions: criticism qualitative for lack variance on the DV

distinctive traits qualitative: positive cases of interest often rare (e.g. war, revolution), more negative cases (e.g. no war, no revolution) + many cases where causes and DV are absent (but these are not really interesting > will not be the basis of case studies)

Question 8

a tale of two cultures

7. Weighting observations

Answer 8

qualitative = “detective” method: fact gathering, experience with similar cases, knowledge of general causal principles = not all pieces of evidence count equally for building an explanation
A theory is usually one critical observation away from being falsified
given my prior theoretical beliefs, how much does this observation affect these beliefs? = no piece of evidence counts more than other parts
Single piece of data can affect posterior beliefs (e.g. Can show a key variable was wrongly measured, when correctly measured the theory can no longer make sense)

• causal-process observations : sort through data with preexisting theoretical beliefs

quantitative = no assumption that some pieces of evidence should count more heavily than others + single observation can’t lend decisive support or critically undermine a theory: only a pattern can do so

• data-set observations (cases / observations)

both methods make sense: if you want to explain a specific outcome it makes sense to go back and forth between theory and observation + if you look into estimate average causal effects it makes no sense to move constantly back and forth between theory and data

Question 9

a tale of two cultures

8. substantively important cases

Answer 9

quantitative = ex ante no important cases (all cases are equal), ex post outliers and influential cases should be examined

qualitative = not all cases are equally important (prior theoretical knowledge makes certain cases especially interesting and theoretically important)

• interest in substantively important cases: cases of normative interests because of a past/current major role in domestic or international politics
• when a theory explains many cases well, but fails to explain one “important” case, it is often called into question (e.g. realism lack of explanation end cold war)

Question 10

a tale of two cultures

9. lack of fit

Answer 10

qualitative = if a case does not conform to the causal model it is not ignored but its distinctive causal pattern is discussed

quantitative = failure to explain a specific case not a problem as long as the estimates for the population as a whole are good = exclusion of idiosyncratic factors (distinctive factors)

= diff approaches to dealing with lack of fit
-> qualitative thinks that prediction error of quantitative should be explained rather than acknowledged
-> quantitative wonders why qualitative spends so much time on research that is not generalizable + may think fully explaining an outcome is an utopian goal

Question 11

a tale of two cultures

10. concepts and measurement

Answer 11

qualitative = clear and precise definitions to achieve conceptual validity to avoid measurement error + try to avoid conceptual stretching

quantitative = focus on operationalization and use of indicators (bc that is for them the major cause of measurement error)

diff leads to skepticism between the traditions
qualitative thinks quantitative indicators are to simplistic, that they omit key elements of definitions + don’t measure the same thing across diverse contexts

= different approaches to measurement error

• qualitative focuses on conceptual validity and wants to eliminate measurement error
• quantitative focuses on indicators and measurement validity and wants to model measurement error and prevent systematic error

Question 12

data set observations vs causal-process observations: the 2000 US Presidential elections

difference CPO and DSO

Answer 12

DSOs = Data-set observations

• data set of quantitative scholars
• standard regression techniques + variants on regression

CPOs = Causal-process observations

• diagnostic “nuggets” of data that make a strong contribution to causal inference
• process tracing to search for CPOs
• tries to approach the problem in diff ways, cross-checking info, asking if something is probable/possible given what we know of diff sources = attention to detail

process tracing seeks to establish the physical and social processes though which puported causes affect outcomes

Question 13

data set observations vs causal-process observations: the 2000 US Presidential elections

2000 US presidential election Florida - DSO vs CPO

Answer 13

Bush votes were lost in the 10 penhandle counties of Florida bc networks declared Al Gore the winner in Florida after the polls had closed in eastern Florida but before they had closed in the penhandle counties

DSO - difference in difference method

Lott: at least 10.000 votes were lost for George W Bush = based on DSO regression analysis: looked at difference between one set of counties that got a treatment in 2000 (penhandle counties whose polls were stil open when the election was called) and those that did not, while controlling for differences reflected in the data from previous elections
-> media ended up suppressing the Republican vote

• assumes that turnout in 2000 can be predicted by turnout in past years after adjusting for idiosyncratic factors (affect each county the same way over time period but vary from county to county + factors that affect all counties in a given year but vary over years)
• performs badly when idiosyncratic factors vary both by county and over time
• Lott used a .10 level of significance rather than .05

CPO -> upper bound for Bush’s vote loss was 224 and actual vote loss probably between 28 and 56 votes

• no more than 1/12 voters in Florida come to the polls in the last hour -> only 1/72 hadn’t voted when the media call was made = 4200 voters could have been swayed by the media,
• but how many even heard it? maybe approx 20% of the 4200 voters saw the report -> max 840 people affected
• many prospective voters were Democrats or Independents and would not have voted for Bush, approx 560 of the 840 would have voted Bush (based on average 2/3 of penhandle voting Bush) and thus could have been affected
• how many of those decided not to vote?10% (based on general knowledge voting behavior)
• • it is just as likely that a Gore voter decided not to vote: for them the elections are also no longer relevant -> smaller net effect

conclusion: CPO show that it is implausible for the media effect suggested by Lott’s analysis to have occurred (Lott’s analysis cannot be valid)

even in a data-rich domain CPO is relevant and valuable

it is necessary to pay attention to the causal processes underlying behavior