1. Demonstrate understanding of concepts

Question 1

Confirmation bias:

Answer 1

tendency to give more weight to evidence that confirms our beliefs than to evidence that challenges them

Question 2

Deductive reasoning:

Answer 2

Deduction begins with abstract theory and assumptions Economics relies heavily on deduction. Useful because does not require collecting observations like inductive reasoning does fast cheap BUT… just because logically consistent does not make the conclusion true.

Question 3

Induction:

Answer 3

1. We use deduction to derive (state) a TESTABLE hypothesis 2. We use induction, to test that hypothesis of what we expect to observe against empirical observations of what actually occurred. begins with observations about the world.

Question 4

Bias against negative results:

Answer 4

There is nothing wrong, defective or unimportant about a study that fails to reject H0 Such studies are just as important as those supporting an effect
or a causal relationship. (Don’t say it ‘didn’t work.’)

E.g. Studies showing that a treatment or drug is ineffective fail to reject H0. There is unfortunately systematic bias against negative results Studies that get a negative result (uphold H0) tend not to get published and tend to be ignored by the public and policymakers e.g. Homeopathy is proven to be ineffective e.g. Trade liberalization does not cause lax environmental regulations

Question 5

Independent and dependent variables:

Answer 5

The DV (Dependent Variable) measures the outcome, whatever is being caused. (We only look at one DV at a time.) aka “Y”. An IV (Independent Variable) measures one factor which is a possible cause. aka “X”. A single DV can and often will have many IVs. We test hypotheses to see which possible causes appear to be real causes and which are not.

Question 6

Falsifiability/unfalsifiability

Answer 6

What makes something unfalsifiable: If there is no possible observation which could be inconsistent the proposition/hypothesis. If you can’t generate a prediction about what we expect to observe in any particular instance.  Unfalsifiable = not scientific. Example: Whether the strike is put down by the government or workers win their demands, either outcome is consistent with the statement So hypothesis cannot be disconfirmed Also, can’t make a prediction about outcome of any specific strike. A falsifiable hypothesis requires separate, independent measures of the IV and DV E.g. explanations for sports victories
“The Giants just wanted it more.” How do we know they wanted it more? Because they won… This measures the DV based on the IV.

Question 7

Internal validity:

Answer 7

Refers to the causal link between independent variables (which, for example, describe the participants or features of the service they receive) and dependent variables (particularly the outcomes of the program). Internal validity is concerned with whether the program is the agent responsible for observed effects, rather than external conditions, artifacts of the methodology, or extraneous factors. It indicates whether the relationship between program inputs and observed outcomes is causal. High internal validity when we can isolate the treatment that caused the outcome. If our design lacks internal validity, then NO • Threats to inference = Threats to internal validity

Question 8

External validity (generalizability):

Answer 8

Concerned with whether the findings of one evaluation can be generalized to apply to other programs of similar type. If this one AIDS prevention program is successful, can we generalize this to other similar kinds of AIDS prevention programs? How far can we generalize the findings? To which class of AIDS prevention programs will they apply?

Question 9

Problems with research question:

Answer 9

• Variation is required in both the IV and DV
• History doesn’t vary and if you introduce a counterfactual then it’s speculative
• Unmeasurable or hard to operationalize, like “power”
• Things that are hard to observe like “motives” and “norms”
• Telegolical – or will assume that the future will behave like the past
• Functionalist – Attributes personal needs/intentions to abstract constructs like society or the world Functionalist theory in IR – countries succeeded in cooperating because they needed to solve a problem
• Mismatch of units of analysis

Question 10

Counterfactuals:

Answer 10

A counterfactual explanation describes a causal situation in the form: “If X had not occurred, Y would not have occurred.” Counterfactuals are not empirical, unless you have comparable cases for which the event did NOT occur - We can’t collect empirical evidence about a state of the world that did not happen, especially without comparable cases where the other outcome occurred. The hypothesis must be falsifiable, meaning others should be able to do the same test with the same methods and achieve the same results so that the hypothesis doesn’t rely on our opinions but the methods we employ

Question 11

Random selection/sampling versus purposive sampling:

Answer 11

A purposive sample is where a researcher selects a sample based on their knowledge about the study (non-representative) and population. The participants are chosen based on the purpose of the sample, hence the name. Random sampling is a part of the sampling technique in which each sample has an equal probability of being chosen. A sample chosen randomly is meant to be an unbiased representation of the total population. … An unbiased random sample is important for drawing conclusions.

Question 12

Randomization (random assignment to treatment and control groups):

Answer 12

In experimental research, random assignment is a way of placing participants from your sample into different treatment groups using randomization. With simple random assignment, every member of the sample has a known or equal chance of being placed in a control group or an experimental group. It prevents the selection bias and insures against the accidental bias. It produces the comparable groups and eliminates the source of bias in treatment assignments. The main purpose of random assignment of treatments, or of the order of treatments, is to even out confounding variables across treatments. By doing this, a cause-and effect conclusion can be inferred that would not be possible in an observational study. With randomization to treatments, the range of values for confounding variables should be similar for each of the treatment groups

Question 13

The types of variables (e.g. nominal):

Answer 13

Nominal aka categorical  for classification e.g. gender, countries, war/not war, yes/no on referendum. Frequency of cases, descriptive categories cannot rank or order values of the variable, so cannot assign a numeric value Common in IR.

Ordinal ranking, not distance no units e.g. social class – lower class, middle and upper e.g. Levels of education (not years) e.g. 1st, 2nd, 3rd in a race. Unlike nominal, can rank but cannot attribute a precise measure or number - e.g. high school, college, grad school - can’t put a linear scale or graph  can’t express as a number because no units.

Ratio e.g. income, battlefield deaths, most economic data. continuous can express as a number can plot on a graph - can compare ratios, so can say “twice as much income” or “20% fewer casualties.”

Question 14

Null hyotheses for testing:

Answer 14

We start by identifying a null hypothesis H0 that there is no causal relationship between the IV and DV. Establishing a relationship between variables proceeds by rejecting H0 - Only if we can reject the null hypo, can we find support for our alternative hypothesis (if it goes in the same direction as the data). The null hypothesis H0 - i.e. There’s no relationship between the variables. - i.e. IV has no effect on the DV Then we see if the data justify rejecting H0 or failing to reject H0. If we reject H0, there is a correlation (or association) between the variables. If we fail to reject H0, there is no correlation.

If we reject the null hypothesis, then, if our initial hypothesis goes in the same direction as the data, we can say the hypothesis is “consistent with the data” “supported by the evidence” “confirmed, not disconfirmed, by the results”

If we reject the null hypothesis, but our initial hypothesis goes in the opposite direction to the data, we can say we it is “inconsistent with the data” “not supported by the evidence” “disconfirmed by the results”  Still a relationship, but opposite of what we expected initially.

Question 15

Double barrelled hypothesis:

Answer 15

A hypothesis with two IVs is double barreled hypothesis  That makes it impossible to know which IV caused any observed change in the DV, so basically useless. A confusing and poorly designed hypothesis with two independent variables in which it is unclear whether one or the other variable or both in combination produce an Skipping breakfast and drinking alcohol are associated with weight gain If observe weight gain, don’t know which factor caused it. Skipping breakfast? Alcohol? Only both together? Hence, useless as hypothesis test.

Question 16

Teleological/functionalist arguments:

Answer 16

Telegolical – or will assume that the future will behave like the past. A teleology is something directed by an ultimate purpose or goal. It can take two forms. First, it is associated with an event that occurs because it is in “God’s plan” or in some overarching, mysterious unseen and unknowable force It is a teleology to say that something occurs because it is part of the “natural unfolding” of some all-powerful inner spirit or Geist (German for spirit). Thus, it is a teleology to say that a society develops in a certain direction because of the “spirit of the nation” or a “manifest destiny. Teleology violates the temporal order requirement of causality. There is no true independent variable because the “causal factor” is extremely vague, distant, and unseen

Functionalist – Attributes personal needs/intentions to abstract constructs like society or the world Functionalist theory in IR – countries succeeded in cooperating because they needed to solve a problem

Question 17

Dyads as a unit of analysis:

Answer 17

A lot international affairs data deals with countries as the unit of analysis (one or more countries) But some phenomena are also studied by collecting observations on dyads – pairs of countries Examples – international conflict, interstate wars, flows (trade, aid, investment, migration), treaties.

Question 18

Mismatch in levels of analysis

1. Ecological Fallacy:
2. Reductionism:

Answer 18

1. Ecological Fallacy: If 10% of Egyptians support the military leadership and 10% of Egyptians are Christian, then it means that Egyptian Christians support the military.” -WRONG – can’t make that claim without observations at the individual, not aggregate, level.

The ecological fallacy arises from a mismatch of units of analysis. It refers to a poor fit between the units for which we have empirical evidence and the units for which we want to make general statements Ecological fallacy occurs when we gather data at a higher or an aggregated unit of analysis but want to say something about a lower or disaggregated unit. It is a fallacy because what happens in one unit of analysis does not always hold for a different unit of analysis. Thus, when we gather data for large aggregates (e.g., organizations, entire countries) and draw conclusions about the behavior of individuals from those data, we are creating an ecological fallacy.

Researchers have criticized the famous study Sui- ~lde {[18971 1957) by Emile Durkheim for the ecologIcal fallacy of treating group data as though they were individual-level data. In the study, Durkheim compared the suicide rates of Protestant and Catholic districts in nineteenth-century western Europe and explained observed differences as due to dissimilarity between people’s beliefs and practices in the two religions. He said that Protestants had a higher suicide rate than Catholics because the Protestants were more individualistic and had lower social integration. Durkheim and early researchers had data only by district. Because people tended to reside with others of the same religion, Durkheim used group-level data {i.e., region) for individuals.

Reductionism: Mirror image of ecological fallacy. Trying to explain national level phenomena with reference to individual level data - e.g. Saying WWI was caused by the assassination of the Archduke. This overlooks higher level factors.

An error in explanation in which empirical data about associations found among small-scale units of analysis are greatly overgeneralized and treated as evidence for statements about relationships among much larger units. Thus, the ideas that physical bodies are collections of atoms or that a given mental state (e.g., one person’s belief that snow is white) is identical to a particular physical state (the firing of certain neurons in that person’s brain) are examples of reductionism.
Associations on the lower level are irrelevant for determining the validity of a proposition about processes operating on the higher level. As a matter-of-fact, no useful understanding of the higher-Level structure can be obtained from lower-level analysts. … If we are interested in the higher-level processes and events, it is because we operate with the understanding that they have distinct qualities that are not simply derived by summing up the subunits.

Question 19

Mill’s Method of Difference:

Answer 19

• A technique for case selection for case study with multiple cases
• Enables some control for confounding variables
• Select cases to hold some important independent variables constant, but show variation on the independent variable of interest
Example:
• The implicit foundation for “area studies,” i.e. Latin American studies.
• The assumption is that countries within one region share many similarities, and that these similarities are related to similar outcomes and not related to dissimilar outcomes

Question 20

Overdetermination problem:

Answer 20

Overdetermination occurs when a single-observed effect is determined by multiple causes, any one of which alone would be sufficient to account for (“determine”) the effect. That is, there are more causes present than are necessary to cause the effect.

1. From a single case, it is rarely possible to determine which IV or combination of the IVs present produced that outcome
   - In case study, we try to eliminate/rule out possible causal factors
   - Can do this if a possible factor is absent in a given case
   - If we find the presence of several factors (IVs), each sufficient to have caused the outcome…
2. It is probably not possible to rule out any causal factors (IVs) which were present

• Example – Explaining low level of development in a single country, which is landlocked, has malaria, soil erosion and low literacy

Question 21

Paradox of uniqueness

Answer 21

a single case study can’t explain a unique case
- can’t say anything about IVs unique to that country causing low development
- If a single case, have zero variation on IVs unique to that country
– need large sample to test impact of unique factors
Historians of the United States and scholars of the US like to claim that the US is exceptional, that it can’t be explained by theories that apply to other countries
🡪 Can’t support this claim unless study US in comparison to other countries
🡪 Flat assertion of uniqueness is not evidence

Question 22

Assessing impact of a particular IV:

Answer 22

• How would you determine impact of…
- Canadian policy on human rights policy in Cuba?
• Virtually impossible to do
• Why? Because so many other factors operating
• All we can observe is the net effect on the outcome of all the IVs, together
• This will be problem anytime you have a single case, particularly if you have no comparable cases (no control group)
– Some IVs will push the outcome one way, others will exert a countervailing pressure the other direction

Question 23

Degrees of freedom: A mathematical rule

Answer 23

You must have more observations (n) than IVs, ideally, lots more (n > # IVs) to measure impact of one IV
• Degrees of freedom is the extent to which n exceeds your number of IVs
• If d.o.f. not > 0, theory is indeterminate
• Indeterminacy in explanation: Cannot assess impact of one particular IV if you have more IVs than observations (n)
• Similar to problem of double barreled hypothesis

Question 24

Confounding factors:

Answer 24

The true causal effect X ➔ Y is “mixed” with the spurious correlation between X and Y induced by the fork X f- Z ➔ Y. For example, if we are testing a drug and give it to patients who are younger on average than the people in the control group, then age becomes a confounder-a lurking third variable. If we don’t have any data on the ages, we will not be able to disentangle the true effect from the spurious effect.

Question 25

Control variables:

Answer 25

To isolate the impact of any single IV, you would need a large n multivariate model of growth including all the major causal variables Including
all the major causal variables isolates the impact of each IV by controlling for the effect of all the others (what multivariate statistics does). The problem with too many control variables: Controls give the feeling of specificity, of precision …. But sometimes, you can control for too much. Sometimes you end up controlling for the thing you’re trying to measure.

Question 26

Selection on the dependent variable:

Answer 26

refers to the practice of restricting one’s set of observations to cases in which some phenomenon of interest has been observed and excluding from the set cases in which the phenomenon was not observed. E.g., when looking at what 50 millionares did, and explaining what might make you rich, you forgot to include millions of other people who did the same thing and did not get rich haha.

1. Selecting observations based on a single value of the DV (no variation)
   - You cannot draw a correct conclusion from such data (no matter how large n)
   🡪 there’s actually no correlation, because there’s no variation on the DV
   - e.g. First slide looking only at heroin addicts
   • If want to explain when wars, revolutions etc are likely to occur, can’t just look at cases in which wars etc have occurred
   – Much better to have a hypothesis about an IV and select cases for variation on that IV
   – can select on DV to generate a hypothesis but not to test one
2. Truncated range of variation on the DV
   🡪 This will distort the true relationship between the variables
   🡪 will produce an incorrect estimate of impact (Shively)
3. Selecting observations for variation on the DV (instead of variation on the IV)
   🡪 This will distort the true relationship between the variables
   - selecting observations based on variation on the IV does NOT cause this problem (Shively)

Question 27

Chi square test statistic:

Answer 27

• Chi square is a test statistic for a 2X2 table. The chi square value will tell us the level of statistical significance (as p) of the correlation in the sample
• The p value is the likelihood that we could observe this correlation in a sample for which H0 was true.

• Actual values are those observed in our sample.
• Expected values (in parentheses) are what we would expect to see if the sample reflected the H0.
• We find expected values by looking at proportions in bottom row
– If IV doesn’t have an effect, proportions the same for each row
• If find zero correlation, this is consistent with H0

Question 28

Strengths/weaknesses of case study method

Answer 28

•	Good for generating new hypotheses
•	Require less data than large n
•	Yields insight into unique events
Weaknesses: 
•	Overdetermination
•	Cases may not be representative
•	Can’t determine magnitude of IVs
•	Poor control for other variables
•	Omitted variable bias

Question 29

Strengths/weaknesses of large n statistical approaches

Answer 29

• Like physical control (above), it lets us control for the effect of other variables, to isolate the impact of the IV we are interested in
🡪 When true experiment not possible, best possible technique for controlling for other independent variables
• Unlike physical control
– Easier to use
– Uses data more efficiently
Weakness:
🡪 It is not sufficient to show correlation between your IV of interest and the DV
🡪 You have to be able to rule out or control for the effect of other IVs (confounding variables) which are present
🡪 Statistical controls (e.g. Multivariate regression) is inferior to random assign’t
🡪 Quality of statistical control
🡪 Depends on correctly identifying IVs to control for confounding variables
🡪 Risk of overcontrolling (Ezra Klein in Book of Why excerpt)
🡪 Depends on correctly measuring all IVs

Question 30

Statistical significance:

Answer 30

Statistical significance refers to the claim that a result from data generated by testing or experimentation is not likely to occur randomly or by chance but is instead likely to be attributable to a specific cause. Or that our sample is representative of the population we want to study.

Question 31

Type I and II error:

Answer 31

A type I error (false-positive) occurs if an investigator rejects a null hypothesis that is actually true in the population; a type II error (false-negative) occurs if the investigator fails to reject a null hypothesis that is actually false in the population.

Question 32

p value:

Answer 32

: A p-value is a measure of the probability that an observed difference could have occurred just by random chance. The lower the p-value, the greater the statistical significance of the observed difference. P-value can be used as an alternative to or in addition to preselected confidence levels for hypothesis testing.

Question 33

Omitted variable bias:

Answer 33

incorrect estimates of effect of IVs because the model excludes important causal factors (confounding variables)

• A problem with large n also but more acute with case study

Question 34

Use and misuse of historical analogy

Answer 34

• Decision makers need cognitive frameworks to organize their reasoning
• Even those who are knowledgeable about history, can misapply historical analogies
Use Correctly:
get the facts (diagnosis) straight
- clearly distinguish between what is known, presumed and what is unclear
- clearly distinguish similarities and differences between present and the historical case