Quiz 2 POLI 399 Quantitative Research Methods Flashcards
Why is research design important?
purpose: impose controlled restrictions on our observations of the empirical world.
- defines domain of generalizability, causality able to say cause or not. rule out alternative explanations.
- draw causal inferences with confidence.
Causality *
- when talking about causality can never know for sure that one variable causes another. increased confidence if:
1. demonstrate covariation: the cause and effect are moving together consistently and in a patterned way. the IV and DV move together, must match proposition.
2. eliminate sources of spuriousness: eliminate a common cause that is moving both the IV and the DV. The relationship appears to be casual but can’t make a causal claim.
3. establish time order. the IV must come first, cause first. changes in the IV must be present before changes in the DV
Classical Experimental Design
- There is an experimental group and a control group. Experimental groups is given or has the IV and the control group does not.
- Conduct experiments to control for spuriousness and establish time order.
Components of Experimental Design
- Demonstrates causality
- Comparison to determine covariation
- Manipulation to determine when the IV is introduced to assess time order
- Control to look for sources of spuriousness.
Internal Validity of a Research Design
- enables us to infer with reasonable confidence that the IV has a causal influence on DV
- has to do with causality
- increase internal validity, increase confidence in the fact that we have a causal claim
- threats: can be extrinsic or intrinsic
Extrinsic Threats to Internal Validity*
extrinsic: arises from the selection of cases. selection bias that experimental group differs from control group before the experimental group is exposed to the IV. this introduces potential spuriousness. (something is moving IV and DV outside hypothesis)
Ensure groups are equivalent through precision matching, frequency distribution matching and randomization.
How do we ensure groups are equivalent *
extrinsic
Precision matching: impractical. match each person in both groups according to certain characteristics they both share. not randomization
Frequency distribution matching: same proportions of characteristics are present in each group. while the individuals may be different, the group averages are the same.
Randomization (best way): randomly assign people to experimental and control group. need to demonstrate frequency distribution matching. need to have equal chances of being selected but in order to do this you need a large enough sample. can’t bias results if randomization is done properly.
Intrinsic Threats to Internal Validity*
Changes in the cases being studied: people change between pre test and post test
Flaws in measurement: having to do with validity (if indicator isn’t representing the target concept)
Reactive effects of being observed: changes in pre test and post test
-Undermine ability to make a causal claim
1. history: a difference in time between the pre and post tests can affect DV values separate from the introduction of the IV due to that events that occur.
2. maturation: people changing their minds within the pre-post test period. Can be psychological or physical processes that have an affect on DV values independent of IV
3. mortality: participants die or lose interest and therefore selectively drop out.
4. instrumentation: reliability problem because the measuring instruments are not performing in a consistent way.
5. regression effect: regression to the mean. the participant appears a-typical for the pre-test but appears more typical during the post-test. this is external to IV exposure
6. reactivity: test effect. pre-test causes the values to change apart from IV.
Which is not an intrinsic threat to internal validity A. selection bias B. maturation C. instrumentation D. all are E. none are
-Selection bias because it is an extrinsic threat to internal validity. has to do with case selection and selection bias.
Demonstrating causality requires
making comparisons
implementing controls
establishing time order, sequence. IV must come before the DV
Countering intrinsic threats to internal validity *
if groups are truly equivalent what happens to one group should affect the other group in the same way. for example: history: exposed to the same events
this is all contingent on the whether the sampling has been done well or not
Threats to external validity
has to do with generalizability: the degree of applicability to the real world and whether it is possible to generalize to what people in the real world are actually doing.
- unrepresentative cases: people in the experiment don’t match the population. need representative cases.
- artificiality of research setting: less likely to match real life contexts.
- reactivity: intrinsinic threat but also has to do with external validity. people are essentially reacting to being studied. corresponds to a traditional critique of the scientific method: Hawthorne Effect.
- tradeoff between the level of confidence in causality and the level of confidence in generalizability.
Variation on Classic Experiment
- post test only only. exposure to IV during post test to the experimental group.
- avoid causal claim bc no baseline
Quasi-Experimental Design
- Can’t use the word “cause”
- Using statistics to establish controls and comparisons
- weaker causal argument.
- researcher can’t randomly assign observations.
- ex post facto experiment, meaning that the researcher approximates the post-test only control group design through multivariate statistical methods.
- sort cases on values of spuriousness
Control Variables
- test hypothesis using control variables
- involves showing that the IV and DV covary in a consistent way, it is not enough to show empirical association, one must look for other variables that may eliminate or change the observed relationship
- effects are held constant while IV-DV relationship is being examined. if the relationship is true, it does not matter if the control viable is held constant or made to vary.
Types of Control Variables 1.
Sources of Spuriousness
- spurious variables cause both the IV and DV, common cause weakens or disappears and therefore there is no covariation .
- relationship is destroyed
- identify it by logically assessing whether there is something causing both the IV and DV or if there is anyone directly acting on IV/DV.
- partial tables are different, have weaker differences across columns, than the original crosstabulation
- IV that are given characteristics such as age or gender, cannot have sources of spuriousness because nothing can move them
The higher a country’s literacy rate, the more democratic it will be. Plausible source of spuriousness.
Level of economic development
Types of Control Variables 2.
Intervening variables
- presumed causal method, mechanism that mediates the relationship between the IV and DV
- this variable explains why the IV is causing the DV
- intervening variables have to do with the why the causal mechanism appears the way it does
- can’t distinguish whether the variables is spurious or intervening from statistics alone
The lower the peoples incomes, the less interest they have in politics, What is the intervening variable?
attention to news or political alienation
Types of Control Variables 3.
Conditional Variables
- has to do with how generalizable the relationship is.
- affects either the strength of the relationship between IV and DV or the form of the relationship between the two.
- IV may have a predictive function in terms of DV for some people but not all
- examine whether there are some sorts of people for whom the IV will not have the predicted effect on the DV.
1. specify relationship in terms of interest, knowledge or concern
2. specify relationship in terms of time or place
3. specify relationship in terms of social background characteristics - partial tables differ from the original table in two different ways. for example, gaps across one partial table may grow, signifying the relationship got stronger (if seen with a corresponding increase in the chosen measure of association.) and gaps across the other partial table may have fallen (corresponding decrease in the chosen measure of association).
The more people favour public healthcare, less likely they are to vote for a party/candidate on the right. What is a plausible conditional variable?
political knowledge
*ideology cannot be a plausible conditional variable
The older people are the more likely they are to oppose same-sex marriage. What is a plausible conditional variable?
sexual orientation
What are research problems?
- are always questions that display how one concept is related to another concept.
- the goal of a research problem is to maximize generalizability.
Why is generality important in the context of research problems?
- the scientific method has generality as one of the goals.
- the research that one engages in has implications for the sample and the relationship being studied in that specific instance.
- the reason that people care about a research problem is due to its implications such as policy.
