Lecture 3: Small-N Comparative Designs

Question 1

Is causation observable?

Answer 1

Causation is fundamentally unobservable. We can make inferences about causation from the associations that we can observe. We can observe associations only

Question 2

Confounding variables

Answer 2

Definition:
Confounding variables (or confounders) are external variables in a statistical model that correlate with both the dependent variable and the independent variable.

Example:
In a study assessing the relationship between physical activity and heart disease, age could be a confounding variable. Age is associated with heart disease risk (older individuals may have a higher risk), and it’s also associated with physical activity levels (older individuals may be less physically active). Therefore, without proper control, age could distort the perceived relationship between physical activity and heart disease.

Question 3

Dependent variable (definition + example)

Answer 3

Definition: The dependent variable is what you measure or observe to see if it changes when you tweak something else.

Example: If you’re seeing how more study hours might improve test scores, the test scores are your dependent variable. It’s the thing you’re curious about changing.

Question 4

Independent variable (definition + example)

Answer 4

Definition: The independent variable is what you change or control to see if it affects something else.

Example: If you’re seeing how more study hours might improve test scores, the number of study hours is your independent variable. It’s the thing you’re changing to see what happens.

Question 5

Confounder bias

Answer 5

Definition: Confounder bias happens when you’re not sure if the result of your experiment is because of what you changed, or because of some other thing you didn’t think about.

Example: Imagine you’re studying if eating more ice cream leads to more sunburns. You see a connection, but then realize people eat more ice cream and get more sunburns in the summer. Here, the season (summer) is a confounding variable that creates a bias in your results. You can’t be sure if it’s really the ice cream or just more sun exposure causing the sunburns.

Question 6

Collider variable

Answer 6

Definition: A collider variable is one that is influenced by two other variables. Not considering this variable can make the relationship between the two other variables misleading.

Example: Imagine you’re studying whether owning a pet influences happiness. You find that people who own pets and people who don’t own pets have similar levels of happiness. But then, you discover that both these groups have a high income. High income is a collider variable here, as it can influence both pet ownership (people with high income are more likely to afford pets) and happiness (high income can contribute to happiness). If you don’t account for income, you might incorrectly conclude that owning a pet has no influence on happiness.

Question 7

What is the fundamental problem of causal inference?

Answer 7

Since causation is defined counterfactually, it is by definition unobservable

Question 8

Give five different reasons of association between two variables (X and Y)

Answer 8

Chance
X causes Y
Y causes X
A third variable Z causes both X and Y (confounder)
X and Y both cause a third variable Z (collider, if we condition on Z)

Question 9

Difference between Confounder Bias and Collider Bias

Answer 9

Confounder Bias: This happens when a third variable, a confounder, influences both variables you’re studying (X and Y). If you don’t consider the confounder, it can trick you into thinking X causes Y, when really, the confounder is influencing both.

Example: You think more chocolate eating causes more cavities. But you forgot about tooth brushing. People who eat more chocolate might also brush their teeth less often. Here, tooth brushing is the confounder, and if you don’t consider it, you might wrongly think chocolate alone causes cavities.

Collider Bias: This happens when both variables you’re studying (X and Y) influence a third variable, a collider. If you consider (or control for) this collider, it can trick you into thinking X and Y are not related, when they really are.

Example: You think people who exercise more (X) are less likely to have heart disease (Y). But both X and Y influence a third variable, being thin (Z). If you only look at thin people (controlling for the collider), you might wrongly think exercise doesn’t protect against heart disease, because among thin people, those who exercise and those who don’t might have similar rates of heart disease.

Question 10

Importance of Research Design

Answer 10

Research designs are crucial in explanatory research as they help us approximate the counterfactual, which is a hypothetical scenario representing what would have happened if a different choice had been made.

Research designs aim to infer causation from association. They help us understand if one variable (X) influences another variable (Y), based on the observed relationship between them.

These designs also attempt to address the non-causal reasons for observing association. This means they help us consider other factors or variables that might be creating a link between X and Y, ensuring that the inferred causality is as accurate as possible.

Overall, a good research design allows us to make more valid causal inferences, reducing the risk of being misled by confounding or collider biases.

Question 11

What is Comparative Research and its different forms?

Answer 11

Comparative research involves making comparisons to draw conclusions. It is a fundamental aspect of all types of research:

1. Experiments: They compare treatment and control groups.
2. Large-N analyses: These make comparisons between many units, like a nationwide survey comparing attitudes across different regions.
3. Small-N analyses: These compare a smaller number of units. This is often achieved through careful case selection, like comparing different policies across a few carefully selected countries.
4. Case-study research: Here, empirical evidence for the case is compared with expectations from different theories and hypotheses. For example, studying one company’s success and comparing it to predictions made by business theories.
5. Unobservable causal effect: This refers to a hypothetical comparison, as in comparing what happened with what would have happened under different circumstances.

Question 12

What is an Experiment in Research?

Answer 12

An experiment is a type of comparative research that compares treatment and control groups. For example, a medical trial may compare outcomes between a group receiving a new drug (treatment group) and a group receiving a placebo (control group).

Question 13

What are Large-N Analyses in Research?

Answer 13

Large-N analyses are a form of comparative research that compare many units. For example, a nationwide survey that compares attitudes across different regions or populations would be a large-N analysis. The ‘N’ refers to the number of units being compared, which is large in this case.

Question 14

What are Small-N Analyses in Research?

Answer 14

Small-N analyses compare a smaller number of units. This often involves careful case selection to ensure meaningful comparison. For example, comparing different policies across a few carefully selected countries is a small-N analysis. The ‘N’ refers to the number of units being compared, which is small in this case.

Question 15

What is Case-Study Research?

Answer 15

Case-study research involves comparing empirical evidence from a single case or a few cases with expectations from different theories and hypotheses. For instance, studying one company’s success and comparing it to predictions made by different business theories is an example of case-study research.

Question 16

What is the Unobservable Causal Effect in Research?

Answer 16

The unobservable causal effect refers to a hypothetical comparison in research. This involves comparing what happened with what would have happened under different circumstances. This concept is central to the idea of counterfactuals in research design, which is a crucial part of inferring causal relationships.

Question 17

What are Small-N Comparative Designs in Research?

Answer 17

Small-N comparative designs are hybrid research designs that combine the in-depth analysis of individual cases (within-case analysis) with the logic of comparing multiple units (as in large-N research).

Key points to consider with small-N designs include:
- Qualitative vs. Quantitative?: While small-N designs often involve qualitative data, they can also be used with quantitative data. The choice depends on the nature of the research question and the available data.

How big/small is ‘n’?: The ‘n’ in ‘small-N’ refers to the number of cases or units being studied. There’s no strict rule for what counts as ‘small’, but it’s typically fewer than would be studied in a large-N design.

The goal is to select a number of cases that allows for in-depth analysis while also enabling meaningful comparisons.

These designs are particularly useful when the researcher wants to understand a phenomenon in great depth but also wants to draw comparisons across cases to generate more general insights.

Question 18

What is the main goal of Small-N Designs in Research?

Answer 18

Small-N designs aim to approximate the counterfactual situation where only the hypothesized factor (or the value of the explanatory variable) differs, while all other conditions remain the same. This design allows for a deep understanding of specific cases and the ability to compare across them

Question 19

What are the strategies used in Small-N Designs to achieve the main goal?

Answer 19

Strategies include narrowing down the range of potentially relevant factors, testing specific hypotheses rather than very general ones, and ruling out alternative explanations to strengthen the internal validity of conclusions.

Question 20

What are the considerations regarding validity and generalizability in Small-N Designs?

Answer 20

While exploring the internal validity of conclusions is essential, it’s also important to question the external validity and generalizability of findings. As small-N designs focus on fewer cases, the findings may not apply to a broader context.

Question 21

What are the challenges and solutions in Small-N Designs?

Answer 21

Due to the smaller number of units in small-N designs, beneficial statistical properties found in large-N designs may not apply. Selection of cases for comparison becomes crucial, and measurement validity can be a potential issue. To address this, researchers may choose cases through conditioning via blocking or balancing, aiming to keep all factors the same while allowing the main explanatory variable (MEV) to vary across cases.

Question 22

What are the primary goals of Small-N Designs?

Answer 22

Small-N Designs aim to:

Retrospectively account for outcomes of specific cases rather than prospectively estimate average causal effects, which is more about theory testing.

Derive hypotheses from what we can learn about the cases at hand, leading to theory generation.

E.g. Why did France and the UK develop central states quicker compared to Italy/Germany

Question 23

How does deductive logic apply in Small-N Designs?

Answer 23

With deductive logic in Small-N designs, a theoretically-motivated research question guides the selection of cases. Researchers then use these cases to confirm or disconfirm the hypotheses. A common research question in this approach is “does variable X account for outcome Y?”

Question 24

How does inductive logic apply in Small-N Designs?

Answer 24

With inductive logic in Small-N designs, researchers start with a set of cases and make inferences about possible causal relationships. This leads to the generation of new theoretical hypotheses, which can then be tested. A common research question in this approach is “what explains outcome Y?”

Question 25

What is the focus of the Most Similar Systems Design I (MSSD I)?

Answer 25

Back: MSSD I focuses on one major hypothesized causal relationship, allowing variation on the main explanatory variable while the values of other possible variables remain constant - applying the logic of blocking and conditioning strategy.

Example: Comparing the influence of tax rates (main explanatory variable) on economic growth in two similar economies, while other factors like population size, GDP, and industry structure remain constant.

Question 26

How are cases matched in MSSD I?

Answer 26

Back: Cases in MSSD I are matched based on relevant characteristics, including confounders and alternative causal factors. Cases differ in the values of the main explanatory variable (MEV).

Example: Comparing two similar countries (like Canada and Australia) for their healthcare outcomes (MEV), matched on similar characteristics such as GDP per capita, population size, and health expenditure, but differing in healthcare system design.

Question 27

What aspects are kept constant in MSSD I?

Answer 27

Back: In MSSD I, everything relevant to the study is kept “constant,” except for the hypothesized factor which varies across cases.

Example: When studying the impact of public education spending (hypothesized factor) on literacy rates in similar regions, other factors like population, language, and culture would be kept constant.

Question 28

What determines the “relevancy” of factors in MSSD I?

Answer 28

Back: The “relevancy” of factors in MSSD I is determined by existing theory and other empirical findings.

Example: If research suggests that literacy rates (outcome) are influenced by public education spending (MEV), household income, and parental education (other factors), these would be the relevant factors in the MSSD I design.

Question 29

What is the logic of blocking and conditioning strategy in research design, specifically in Most Similar Systems Design I (MSSD I)?

Answer 29

Back: The logic of blocking and conditioning strategy in MSSD I involves selecting cases that are similar on all relevant variables except the main explanatory variable. This strategy helps isolate the effects of the main explanatory variable on the outcome.

Example: Imagine studying the impact of a new teaching method on student performance. Two similar classes (similar in class size, age of students, subjects taught, etc.) are chosen. The new teaching method is applied to one class (this changes the main explanatory variable) while the other continues with the old method. The ‘blocking’ involves selecting two similar classes, and ‘conditioning’ is applying the new teaching method to one of them to see its effects.

Question 30

How does MSSD I interpret outcomes in relation to the main explanatory variable (MEV)?

Answer 30

In MSSD I, if the outcome is the same across cases despite variation in the MEV, it suggests that the MEV is not causing the observed outcome. However, if the outcomes differ, the hypothesis that the MEV is not the cause cannot be rejected.

Example: Suppose we’re studying the impact of tax rates (MEV) on economic growth in two similar economies. If both economies show similar economic growth despite different tax rates, it suggests that the tax rate (MEV) is not causing the economic growth. If the economies show different growth rates, we can’t reject the possibility that tax rates influence economic growth.

Question 31

What are the limitations of Most Similar Systems Design I (MSSD I)?

Answer 31

The main limitations of MSSD I are:

Small measurement errors and/or random variability can lead to incorrect conclusions. This is because MSSD I relies on precise measurement and a high degree of similarity between cases.

The design cannot accommodate complex relationships such as equifinality (different initial conditions leading to similar effects) and conjunctional causation (multiple conditions jointly causing an effect).

Example: If studying the impact of educational spending (MEV) on literacy rates in two similar regions, small errors in measuring spending or literacy can lead to incorrect conclusions. Similarly, if literacy rates are influenced by a combination of factors (like teacher quality, parental support, and school infrastructure) acting together, MSSD I would have difficulties accommodating these complex relationships.

Question 32

How can the limitations of Most Similar Systems Design I (MSSD I) be addressed?

Answer 32

The limitations of MSSD I can be addressed in the following ways:

By adding more cases: This can enhance the robustness of the study, increase the sample size, and improve the validity of the conclusions.

Alternatively or complementarily, by adding within-case evidence to evaluate hypotheses: This involves gathering and analyzing more detailed data within each case, which can provide deeper insights and potentially identify factors or relationships not captured in the initial design.

Example: If we’re studying the impact of tax rates (MEV) on economic growth, we could add more similar economies to our study to address limitations. Alternatively or in addition, we could dig deeper into each economy’s case, analyzing aspects such as specific sectors affected by tax rates, changes over time, etc., to provide richer evidence for our hypothesis.

Question 33

What is the focus of Most Similar Systems Design II (MSSD II)?

Answer 33

MSSD II focuses on both the values of the control variables and the outcome variable. In this design, cases are chosen that are as similar as possible on the control variables, but differ in the outcome of interest.

Example: If studying the impact of different healthcare systems on life expectancy, similar countries (in terms of economic status, education levels, and other control variables) would be chosen that have different life expectancies (outcome of interest). This allows researchers to examine the differences in healthcare systems in relation to the differences in life expectancy.

Question 34

What is the procedure and logic of Most Similar Systems Design II (MSSD II)?

Answer 34

In MSSD II:

1. We already know that the outcome variable differs across cases.
2. Cases are chosen that are similar on relevant characteristics, but the Main Explanatory Variable (MEV) is unknown.
3. The goal is to search for a variable or a difference that accounts for why similar cases have different outcomes.
4. The logic is to find something that is NOT common between the cases, because anything common cannot be responsible for the difference in outcomes.
5. It is a bottom-up, hypothesis-generating approach, not hypothesis-testing.

Example: Suppose we are studying different countries that have similar socio-economic profiles but different levels of environmental sustainability. We know the outcome (environmental sustainability) differs, and we’re looking for a variable (which is not common between these countries) that might explain this difference. This could lead to generating a hypothesis, for example, about the role of environmental regulations in shaping a country’s sustainability outcomes.

Question 35

What is the procedure and logic of Most Different Systems Design (MDSD)?

Answer 35

In MDSD:

1. The goal is to find very different cases that share the same outcome.
2. The aim is to find what is common between these cases that can account for their shared outcome.
3. MDSD is essentially the mirror image of MSSD II.

Example: Suppose we are studying countries with high environmental sustainability but which are very different in terms of socio-economic profiles (like GDP, literacy rates, population size etc.). We aim to identify a common factor amongst these different countries that might explain their shared outcome of high environmental sustainability. This could lead to identifying a shared aspect, such as stringent environmental regulations or a strong cultural emphasis on sustainability.

Question 35

What is the procedure and logic of Most Different Systems Design (MDSD)?

Answer 35

In MDSD:

1. The goal is to find very different cases that share the same outcome.
2. The aim is to find what is common between these cases that can account for their shared outcome.
3. MDSD is essentially the mirror image of MSSD II.

Example: Suppose we are studying countries with high environmental sustainability but which are very different in terms of socio-economic profiles (like GDP, literacy rates, population size etc.). We aim to identify a common factor amongst these different countries that might explain their shared outcome of high environmental sustainability. This could lead to identifying a shared aspect, such as stringent environmental regulations or a strong cultural emphasis on sustainability.

Question 36

What is the procedure and logic of Most Different Systems Design (MDSD)?

Answer 36

In MDSD:

1. The goal is to find very different cases that share the same outcome.
2. The aim is to find what is common between these cases that can account for their shared outcome.
3. MDSD is essentially the mirror image of MSSD II.

Example: Suppose we are studying countries with high environmental sustainability but which are very different in terms of socio-economic profiles (like GDP, literacy rates, population size etc.). We aim to identify a common factor amongst these different countries that might explain their shared outcome of high environmental sustainability. This could lead to identifying a shared aspect, such as stringent environmental regulations or a strong cultural emphasis on sustainability.

Question 37

What is Qualitative Comparative Analysis (QCA) and why is it used?

Answer 37

QCA is a research method that formalizes comparisons among a medium number of cases. It is useful when the number of cases is large enough that patterns are difficult to appreciate through pairwise comparisons without more formal tools. QCA relies on Boolean minimization, a system of logic for binary variables.

Example: Imagine you are studying the impact of various policy measures on economic growth across 20 countries. QCA would allow you to systematically compare the combinations of conditions in these countries and identify patterns that are associated with higher or lower economic growth.

Question 38

What are the key features and varieties of Qualitative Comparative Analysis (QCA)?

Answer 38

QCA generalizes the ideas of small-N comparisons based on paired comparisons and uses set theory, where a case “belongs” to the set “outcome observed”, another to the set “outcome not observed”, etc. There are varieties of QCA, including crisp-set QCA (binary variables only), multi-variate QCA, and fuzzy-set QCA.

Example: Suppose we’re studying policy measures for environmental sustainability in various countries. With crisp-set QCA, we might categorize countries simply as having a specific policy (1) or not (0). Multi-variate QCA would allow us to consider multiple policies simultaneously, while fuzzy-set QCA would let us consider degrees of policy implementation (e.g., fully implemented, partially implemented, not implemented).

Question 39

What is the “outcome variable” in a research study?

Answer 39

The outcome variable, also known as the dependent variable, is the variable that the researcher wants to predict or explain in a study. It’s the variable of interest and is dependent on other variables known as predictor or independent variables. Changes in the independent variables are hypothesized to cause changes in the outcome variable.

Example: If you’re studying the impact of study hours on test scores, the “test scores” would be the outcome variable - it’s the variable you’re interested in and hypothesizing will change based on the number of study hours (the independent variable).

if you’re studying the effect of study hours (X) on test scores (Y), test scores (Y) would be the outcome variable because you’re hypothesizing that it will change depending on the number of study hours (X).

Question 40

What is the Main Explanatory Variable (MEV) in a study?

Answer 40

The Main Explanatory Variable (MEV), also known as the independent variable or predictor variable, is the variable that the researcher manipulates or changes to observe its effect on the outcome variable (dependent variable). It is the presumed cause in a cause-and-effect relationship being studied.

Example: If you’re studying the effect of exercise duration (X) on weight loss (Y), the “exercise duration” would be the MEV - it’s the variable you’re changing to see how it impacts the outcome variable, weight loss.

Question 41

What are the concepts of causality in Qualitative Comparative Analysis (QCA)?

Answer 41

In QCA, there are two key concepts of causality:

1. Conjunctional or combinatorial causality: Causal factors are not relevant in isolation but are significant in certain combinations. It emphasizes that context matters when examining causes.
2. Equifinality: It means that there are multiple pathways or different processes that can produce an outcome. Multiple combinations of factors can lead to the same result.

Both these ideas are captured through the logic of necessary and sufficient conditions.

Example: If we’re studying the success of startup businesses, conjunctional causality could mean that a combination of factors like sufficient funding, an experienced team, and a strong business plan together influence success, rather than any of these factors alone. Equifinality means that one startup might achieve success through a unique product offering while another might succeed due to an effective marketing strategy - there are multiple paths to the same outcome.

Question 42

What are the concepts of Necessary and Sufficient Conditions in a research context, and what does INUS stand for?

Answer 42

In research:

1. Necessary Condition: A condition that must be present for the outcome to occur. If the condition is absent, the outcome will not happen.
2. Sufficient Condition: A condition that, if it is present, guarantees the occurrence of the outcome.
3. INUS: An acronym for ‘Insufficient but Necessary part of an Unnecessary but Sufficient condition’. It refers to a condition that is necessary but not enough on its own to produce an outcome; however, it’s part of a condition that is enough to produce the outcome, even though that larger condition isn’t the only one that could do so.

Example: To get a driver’s license (outcome), being of legal age might be a necessary condition - you cannot get a license if you’re not of the legal age. Completing a driving test successfully might be a sufficient condition - if you do this, you will get your license. For INUS, consider passing a written test. On its own, it’s insufficient to get a license, but it’s necessary as part of the larger condition of ‘completing all tests successfully’, which is sufficient for getting a license, even though it’s not the only way (for instance, you might also need to complete a vision test).