Lecture 4 - Quantitative Research Design

Question 1

What is a research design?

Answer 1

Research design is the overall strategy that you will use to answer your research question.

• Some questions just cannot be answered with certain designs
• Fit between RQ and design is crucial (expect critical questions about this at the exam)

Question 2

What are the difference between research design and research methods?

Answer 2

Design is the plan, methods are the tools

Question 3

What does research design include?

Answer 3

• Unit of observation - depends on the phenomenon we want to study - our dependent variable (DV)
• Condition/treatment/independent variable which will influence DV
• Level of analysis - depending on the theoretical framework where does the phenomenon of interest lie?

Question 4

Can you explain the research strategi: Formal theory and literature reviews from McGraths model? (Mcgrath)

Answer 4

In both approaches, researchers often summarize the literature in an area of research in order to conceptualize models for empirical testing.

Theory often involves an inductive process, as described by Dubin (1976). An example of a theoretical piece is the 1996 work of Lei, Hitt, and Bettis on how to develop competitive advantage. The purpose of their research was “to build and explain an integrative model of the development and outcomes of dynamic core competencies’’ (Lei et al., 1996: 550).

Literature reviews often employ a deductive process that generally provides researchers with hypotheses for empirical testing (Dubin, 1976). However, they may also propose new theories based on inductive conclusions

Question 5

Can you explain the research strategi: The sample survey strategy from McGraths model? (Mcgrath)

Answer 5

As a research strategy, the sample survey maximizes the representative sampling of the population units studied.

Question 6

Can you explain the research strategi: The laboratory experiment strategy from McGraths model? (Mcgrath)

Answer 6

The laboratory experiment brings participants into an artificial setting for research purposes (Meltzoff, 1998). An attempt is usually made to create a universal setting that will not have a significant effect on the results.

Question 7

Can you explain the research strategi: The field study strategy from McGraths model? (Mcgrath)

Answer 7

The field study investigates behavior in its natural setting. Obtrusive primary data collections involve data that are collected by researchers.

Question 8

Can you explain the research strategi: The field experiment strategy from McGraths model? (Mcgrath)

Answer 8

A field experiment involves collecting data in a field setting but manipulating behavioral variables. For example, Earley (1986) conducted a field experiment in which 36 managerial trainees from either the United States or England participated in a study assessing different methods of supervision. This research strategy is moderately high on precision of measurement (and control of behavioral variables) and realism of context but low on generalizability.

Question 9

Can you explain the research strategi: Compute simulation strategy from McGraths model? (Mcgrath)

Answer 9

Computer simulation involves artificial data creation or simulation of a process. One method used is the Monte Carlo method, a technique in which an estimate of a parameter is obtained by random sampling. Researchers often use such a technique when it is difficult to obtain an analytical solution (Nunnally & Bernstein, 1994).

Question 10

Can you explain the research strategi: Triangulation strategy from McGraths model? (Mcgrath)

Answer 10

Increased triangulation should improve the ability of researchers to draw conclusions from their studies. The use of a variety of methods to examine a topic might result in a more robust and generalizable set of findings (higher external validity).

Forms of triangulation

• Sources
• Data types
• researcher
• Theory

Question 11

How do you test your quality of your research design? (Scandura)

Answer 11

We evaluate validity using various forms of validity

Different desings score different for different forms of validity

Question 12

What is internal validity? (Scandura)

Answer 12

Refers to the degree to which a design is able to make strong inferences about causal relationships between the IVs and DV.

Can we establish causality between a predictor and an outcome (IV and DV)?

We cannot observe causality, we infer causality.

Question 13

What is an example of high internal validity? (Scandura)

Answer 13

Laboratory experiments, for example, have high potential internal validity based on their precision and on the control of behavioral variables, as do longitudinal studies, in which cause and effect may be established.

Question 14

What are causality? (Scandura)

Answer 14

Change in one variable causes the change in another variable, other things being equal.

Question 15

What are the conditions that must be satisfied for causality?

Answer 15

1. Empirical association - correlation
   - If IV goes up so does the DV and vice versa
   - This association has to be substantial (we often test this through statistical test)
2. Temporal precedence of the IV
   - Cause must come before effect
3. Absence of alternative explanations- nonspuriousness
   - Lack of other viable explanations for the relationship
   - Example there might be other correlation between the amount of fire and the firemen showing up, it could e.g. be the weather etc.

Question 16

What is external validity? (Scandura)

Answer 16

A way to state whether the findings of the study are applicable to the real world. Or to what extent can we generalize the findings? What kind of population does it work to?

External validity can for instance be high with formal theory and sample surveys.

Question 17

What is construct validity? (Scandura)

Answer 17

The degree to which the measures we use represent how well the phenomena we are interested in?

Question 18

Why can construct validity be an issue in experiments and surveys? (Scandura)

Answer 18

Experiments
- Manipulation could be inaccurate e.g. does 15 min team building game show team identity well enough?

Surveys (especially an issue here)
- Are a set of 20 multiple choice questions a good presentation of someone’s personality?

Question 19

What is statistical conclusion validity (Scandura)

Answer 19

Is our design powerful enough to statically detect the relationship that we are interested in?

Question 20

What are the two main threats of statistical conclusion validity? (Scandura)

Answer 20

Two main threats:

• Violations of assumptions of statistical tests
• Power-related issues. (Sample size can interrupt this)

Question 21

What is ecological validity (Scandura)

Answer 21

This deals with the question if we would find the same results in a real context? It is linked to external validity, however it is a specific link of it, where we focus on the nature of tasks/stimuli.
- Example; eye-tracking are used to research decision making behaviour of people.

Question 22

McGrath places three ideal point around his/her model of research strategies. Can you name and explain these?

Answer 22

A. Generalizability: high external validity, high statistical conclusion validity (happens in large sample surveys)

B. Precision: high internal validity, high construct validity (experiments i.e. lab-experiments)

C. Realism: High Ecological validity, high external validity (happens in field studies)

Question 23

What does an efficient research design aim to? (Kerlinger)

Answer 23

(1) maximize the variance of the variables of interest
(2) control variance of ‘unwanted’ variables that may affect the experimental outcome
(3) minimize error or random variance

Question 24

What is variance? (slides)

Answer 24

How spread out is the data from the mean

Question 25

How do you maximize the variance? (kerlinger)

Answer 25

• Make sure the treatments are really different
• Make sure that the data collection is occurring in context where variance is present

→ The first thing is to ensure that we can capture the thing we are interested in
→ Collect data in a settings were you expected to find variance in your DV

Question 26

How do control the uninterested variables? (kerlinger)

Answer 26

• Eliminate any variance due to this variable (e.g.only women)
• Randomization → they will never have a systematic effect.
• Build it into design- make it a treatment variable
• Match cases based on a confounding variable
• When none of these are possible - “capture” it- measure it

Question 27

How do you minimize error variance?

Answer 27

• Increase control of conditions

- Increase reliability of measures

Question 28

What is a Quasi-experiment and Quasi randomization?

Answer 28

Quasi experiments refers to experiments that does not have full control over all variables.

Quasi randomization is difference in the groups you include in the experiement

Question 29

What is cross-sectional study and what could it be in terms of the method?

Answer 29

Cross-sectional analysis is focused on data points in a single point in time

In survey or secondary data, observations are made on multiple individuals (or any other unit) at the same time.

Question 30

What could a longitudinal study be?

Answer 30

Survey or secondary data

Multiple observations on multiple individuals (or any other unit) over time.
→ Panel dataset
→ Time series dataset

When observing the same unit(s) over time - Time series.
→ Focus on time trends- maximize effect of time - time is the systematic variance that we are interested in