reading 5 - positivist process tracing : explaining outcomes Flashcards
process tracing from philosophical roots to best practices
introduction (p.1-9) + defining process tracing
increasing emphasis on causal explanation via reference to hypothesized causal mechanisms (combines social and institutional structure and context with individual agency)
- related forms of analysis date back to Greek historian Thucydides
when is something good process tracing?
-> need for rigorous methodological safeguards against inferential errors
process-tracing =
works for both individual and macro-level explanations
- techniques for examining intermediate steps in processes (cognitive psychology US 1960s-70s concept)
- use of evidence from within case studies to make inferences about historical explanations (L. George)
- some argue it is incompatable with rational choice, but it isn’t
-> examination of intermediate steps in a process to make inferences about hypotheses on how that process took place and whether and how it generated the outcome of interest
“histories, archival documents, interview transcripts, and other sources to see whether the causal process a theory hypothesizes or implies in a case is in fact evident in the sequence and values of the intervening variables in that case” + aims to identify the intervening causal process (causal chain/mechanism)
diagnostic evidence = indicates the kind of process taking place
exogeneous variables = variables excluded from a model bc they are not powerful or frequent enough
complementary variable = add/substract from the effects of the main variables of interest
- variable is what causes the outcome
- diagnostic evidence = steps that don’t cause the outcome, but translate the cause to the outcome
-> process tracing not intervening variable but: analysis of evidence on processes, sequences, and conjunctures of events within a case for the purposes of either developing or testing hypotheses about causal mechanisms that may explain the case
!process tracing similar to Robert’s historical explanation with micro-correlation (diff = process tracing more explicit about theories)
process tracing from philosophical roots to best practices
p. 20-31: process-tracing: best practices -> three part standard + 10 best practices
- meta-theoretically = grounded in philosophical base that is ontologically consistent with mechanism-based understandings of social reality and methodologically plural
- contextually = utilize pluralism to reconstruct hypothesized causal processes and keep sight of broader structural discursive contexts
- methodologically = take equifinality seriously + consider alternative causal pathways
-> 10 best practices for systematic, operational, and transparent application of process tracing:
- cast the net widely for alternative explanations
- be equally tough on the alternative explanations
- consider potential biases of sources
- take into account if the case is most or least likely for alternative explanations
- make a justifiable decision on when to start
- be relentless in finding diverse and relevant evidence, make a justifiable decision on when to stop
- combine process tracing + case comparisons when relevant
- be open to inductive insights
- use deduction to ask what process leads to the outcome if your explanation is true
- remember that conclusive process-tracing is good, but not all good process tracing is conclusive
process tracing from philosophical roots to best practices
cast the net widely for alternative explanations
= research design step
evidence more convincing when inconsistent with alternative explanations
-> no alternative explanations offered -> les convincing
- check if major theoretical categories or social explanations have been overlooked
- check explanations offered by regional specialists and functional experts
- check if both structural and agent-based explanations have been considered
process tracing from philosophical roots to best practices
- be equally tough on the alternative explanations
fairness to alternative explanations requires that we fully consider evidence that fails to fit the explanations that interest us most, as well as evidence that fits explanations that initially interest or convince us the least
!still: not all explanations have to be looked into equally deeply: some are quickly undermined by evidence
bad = “first mover advantage” = process tracing on this one explanation and turn to evidence on alternative explanations only to address anomalies that confront the privileged first mover
better = outline process tracing predictions of alternative explanations of a case in advance and test them all against the evidence
process tracing from philosophical roots to best practices
consider the potential biases of evidentiary sources
some agents have instrumental interest in providing evidence that convinces observers that some explanations are stronger than others
how to fix? -> 2-step Bayesian analysis
- attach Bayesian priors to the possible instrumental motives + weigh the evidence those agents give in light of those priors
- use evidence to update prior expectations on their motives, use these updated priors in assessing subsequent evidence
i.e. Given A’s possible motives, how much should I trust what he or she says?
+ diff evidence (e.g. public vs private statements) have diff evidentiary status
+ consider bias in secondary sources (historians often pick sources that confirm their arguments)
process tracing from philosophical roots to best practices
4 take into account whether the case is most or least likely for alternative explanations
most updating of prior expectations on the strength and scope of theory necessary when:
- theory fails to explain a case in which it is most likely to apply
- theory succeeds in explaining a case in which it is least likely to apply
-> process tracing needs o insure that such cases are not flukes, that scope conditions of prior theories need to be revised
(e.g. no revision necessary when most likely case fails to be explained bc one variable that is really rare)
process tracing from philosophical roots to best practices
5 make a justifiable decision on when to start
justifiable starting point depends on how a researcher defines the puzzle/question they are trying to explain
rules of thumb on when to start = critical juncture (institution/practice contingent or open to alternative paths) + consider what earlier and later critical junctures may be relevant as well
OR when a key actor or agent enters the scene or gains capacity
important to show how institutions/practices coudl have reproduced themselves for long periods of time
process tracing from philosophical roots to best practices
6 be relentless in gathering diverse and relevent evidence, but make a justifiable decision on when to stop
not all evidence is equal: the more probative we expect it to be, the more effort we should expend to obtain it
- Bayesion inspired criteria to assess potential probative value of data not yet obtained: smoking-gun, doubly decisive, straw-in-the-wind, hoop test (not explained)
Bayesian logic -> seek diverse and independent streams of evidence (triangulation among diverse data sources)
! be careful that not all sources have the same error/bias -> will lead to ever-more confident in false explanation
when to stop?
Bayesian logic : stop when repetition occurs
process tracing from philosophical roots to best practices
7 combine process tracing with case comparisons when useful for the research goal and feasible
single-case or no-variance designs = can have more general implications for theories: e.g. when they disprove claims of necessity or sufficiency
for many inferential purposes comparative case studies can be more powerful sources of inference than single-case designs
most-similar case comparison -> process tracing can establish that other differences between cases do not account for the similarity in their outcomes
process tracing from philosophical roots to best practices
8 be open to inductive insights
process tracing -> researcher can stumble upon many potential causal factors not anticipated based on hypotheses -> opportunity to rethink prior explanations
process tracing from philosophical roots to best practices
9 use deduction to ask “if my explanation is true, what will be the specific process leading to the outcome?”
clarify facts and sequences within a case that should be true if each of the alternative hypothesized explanations is true
= check against confirmation bias
!also with inducted new explanations: think of observable implications that must be true and are independent from the evidence that gave rise to the explanation
process tracing from philosophical roots to best practices
10 remember that conclusive process tracing is good, but not all good process tracing is conclusive
! be transparant about process tracing: acknowledge uncertainty that remains when evidence does not allow high levels of confidence in supporting some hypotheses and discounting others
- e.g. when document shave been destroyed or remain classified or participants don’t want to do an interview
more continuous narrative explanation of a case + closer fit evidence with some explanations and not others -> more confidence in explanatory inferences based on proces-tracing
evidentiary tests in process tracing
probative value evidence depends on degree to which a hypothesis uniquely predicts that evidence, and the degree to which it is certain in doing so
- hoop test = evidence that is certain but not unique -> failing it disqualifies an explanation + passing it does not greatly increase confidence in the explanation
e.g. accused was in the country the day of the murder - smoking-gun test = unique, but not certain -> passing strongly affirms an explanation, but passing it is not necessary to build confidence in an explanation
e.g. smoking gun in a suspect’s hands right after the murder - doubly decisive test = unique and certain evidence (necessary and sufficient) to provide great confidence in an explanation
e.g. video footage of crime - straw-in-the-wind tests = weak/circumstancial evidence that is neither unique nor certain -> one test not decisive, but series of tests increase confidence in one explanation
!whether a test provides strong evidence in favor of a hypothesis depends on how frequently the pass condition occurs naturally
