4. Quantitative research design Flashcards
What is research design?
Research design is the overall strategy that you will pursue in order to answer your research question. It
ensures coherence between the different steps of the research process, so the study flows in logical
manner
* Set of instructions which rule the gathering and analysis of the data
Research designs differ based on:
* Whether they are supposed to describe or explain (what or why)
* The logic of comparison they utilize
* What type of variance do they explain and what type of variance do they control
* Between units- f.e. individuals, groups, time points? Between units over time?
* The in-built constraints on the quality of the data that they produce
What is the difference between research design and research methods?
Research design is a plan to answer your research question. A research method is a strategy used to implement that plan
Research designs include
* Unit of observation- depends on the phenomenon we want to study
* Condition/treatment/Independent variable which will influence dependent variable
* Number of replications
* Level of analysis- depending on the theoretical framework where does the phenomenon of interest lie? DV and IVs are on
individual or higher level of analysis?
Research design based on what type of variance it is explaining: Experiment, cross-sectional, longitudinal
Research design is not the same as method of data analysis
Experimental study can utilize within the scope of one
project
* ANCOVA- to compare results of treatment and
control group
* Factor analysis- to create scale from
administered questionnaire
* Logistic regression- to predict probability of
individual results within treatment and control
group
Name 3 authors that have different views on research design
McGrath - Typology of research strategies
Saunders - Research onion
Bryman and Bell - Table with quantiative and qualitative in experimental, cross-sectional, longtiduinal, case study, comparative
What is experiment?
Random assignment of subjects to
treatment/control group
What is cross-sectional?
Survey or secondary data
Observation on multiple individuals (or any
other unit) at the same time
Sample should be randomly drawn from
population
What is longitudinal?
Survey or secondary data
Multiple observations on multiple individuals
(or any other unit) over time
* Panel dataset
* Time series dataset
Name 5 areas in research design quality
Statistical conclusion validity
Internal validity
Construct validity
External validity
Ecological validty
What is research validity
- the quality criteria
- Informativeness of a specific study for the development and support of the hypotheses
- Each design has in-built flaws
(Cook and Campbell)
Internal validity
- Can we establish causality between the treatment and effect (IV and DV)?
- Experiment is the gold standard when it comes to internal validity
- Controlled environment and manipulation of treatment
- Randomization- selecting random samples from population should produce statistically equal groups, equal probability
distributions of all potential confounders - so if they are assigned to different treatments, the variance in dependent variable should be caused by treatment
since everything else is equal
Why is internal validity important?
Internal validity makes the conclusions of a causal relationship credible and trustworthy. Without high internal validity, an experiment cannot demonstrate a causal link between two variables
- It is the difference between descriptive and explanatory research (testing theory)
- Can be established only in randomized experiments (strongly)
- All other research designs are compared to the “gold standard” of experiments when it comes to causality and try to achieve
similar level by - Controlling/measuring as many potential cofounding variables as possible
Causality
Change in one variable causes the change in other variable, other things being equal
Conditions that must be satisfied for causality
Empirical association- covariance/correlation
- If IV goes up so does the DV and vice versa
- This association has to be substantial (we often test this through statistical tests)
Which meaning can correlation have between DV and IV
Correlation can have multiple meanings
Examples:
* IV has a causal effect on DV
* DV has a causal effect on IV- reversed causality
* IV and DV have both causal effect on each other
* IV has causal effect on other variables that then influence
DV
* DV has a causal effect on other variables that then
influence IV
* IV and DV have some other causal effects in common
External validity
External validity is the extent to which you can generalize the findings of a study to other situations, people, settings, and measures
- To what extent can we generalize?
- The causal relationship found would apply beyond the sample that is currently studied, beyond the specific
time and specific settings OR would apply to the specific population, time and settings that the theory is
targeting.
Selection of subjects, settings and time matters
* Subjects
* Convenience samples- self-selection bias
* Settings
* Cultural context, industry, size of the company…
* Time
* marketing research before social media
Construct validity
- Do measures we use represent well the phenomena we want to capture?
*Manipulation/treatment (experiments) are fallible operationalizations of the conceptual variable we want to
capture
*Constructs/scales/indexes (survey) are fallible operationalizations of the conceptual variables we want to
capture
Statistical conclusion validity
Is the design precise and powerful enough to detect the relationship between variables if it indeed exists?
- Violations of assumptions of statistical tests
- F.e. analysis of variance assumes normality and equal variances in each group
- Independence of observations- very important in the design stage
