L6 Lab Experimental research Flashcards
Why an experiment?
To test causality: cause and effect relationship among variables
Characteristics of causality
‒ X and Y co-occur (correlation)
‒ A logical explanation for the effect of X on Y is needed
‒ X proceeds Y in time
‒ No other cause (Z) explains the co-occurrence of X and Y
What is an experiment?
Data collection method where one or more IVs are manipulated to measure the effect on the DV, and where you control for other causes
Independent variable
Variable that is manipulated (aka the “treatment” variable)
‒ E.g., price, packaging, degree of advertising, employee bonus, …
‒ Ways to manipulate:
» Presence vs. absence (e.g., bonus vs. no bonus)
» Frequency (e.g., high bonus vs. low bonus vs. no bonus)
» Type (e.g., punishment vs. reward)
Dependent variable “y” or “o”
Variable that is measured
‒ E.g., sales, click-through rate, purchase intention, attitude, motivation, performance,…
‒ Can be nominal, ordinal, interval, or ratio
Extraneous variable
Every possible variable that can influence the DV, other than the IV
‒ E.g., store location, age, gender, culture, …
Internal validity
To what extent does the research design permit us to say that the independent variable causes a change in the dependent variable
External validity
To what extern are the results found in the lab setting transferable or generalizable to actual organizational or field setting.
- Without internal, no external validity
Confound
A variable (Z) that threatens internal validity - roadblock
Lab experiment
Artificial setting to have as much control as possible over the manipulations (incl online experiment)
Field experiment
Natural environment where manipulation is possible
− Problems with randomization
− Problems to exclude external influences
Threats to internal validity
- history effect
- maturation effect
- testing effect
- instrumentation effect
- selection bias effect
- mortality effect
- statistical regression effect
History effect
Events/factors outside the experiment have an
impact on the DV during the experiment
Maturation effect
Biological/psychological of participant changes over time
- growing older
- getting hungry
- getting tires
- getting bored
Testing effect
Prior testing affects the DV
Instrumentation effect
The observed effect is due to a
change in measurement
- different person that gives interview. (same as different teacher)
Selection bias effect
Incorrect selection of respondents (experimental and/or control group)
- not only bonus leads to better work, the other group also is higher educated which you did not know
Mortality effect
Drop out of respondents during experiment
statistical regression effect
extreme scores in the beginning and less extreme in the end
How to increase internal validity
- Randomization
- Design control (Control group and extra group)
- Statistical control
Randomization
Random allocation of participants to different conditions (selection bias, but also instrumentation, history, mortality)
Control group (design control)
Include group that does NOT receive the treatment (history and maturation, but also instrumentation, and statistical regression)
Extra groups (design control)
E.g., groups without pre-test, but with an experimental manipulation (to exclude the effects of pre-testing) (testing)
Statistical control
Measure extraneous variables, and include these in the statistical analysis (covariance analysis) (history and selection bias)
Pre-experimental designs
NO RANDOMIZATION!
- One-Shot Case Study
- One-Group Pretest-Posttest
- Static Group
One-Shot Case Study
- Cause-effect
- X O
- No comparison possible
One-Group Pretest-Posttest
- Investigate influence of a variable
- O X O
- No control for extraneous variables
Static Group
- Comparing a treatment with a control group
- EG: X O1
- CG: O2
- EG and CG can differ
True experimental designs
DO HAVE RANDOMIZATION OF PARTICIPANTS
- Pretest Posttest Control Group
- Posttest-Only Control Group
- Solomon Four- Group
Pretest Posttest Control Group
- Control for confounds
- EG: R O1 X O2
- CG: R O3 O4
- The old workhorse of traditional experimentation
Posttest-Only Control Group
- When pre-tests are not possible
- EG: R X O1
- CG: R O2
- More simple than PPCG – mortality effects possible
Solomon Four- Group
Minimize effects of pre-testing
- EG1: R O1 XO2
- CG1: R O3 O4
- EG2: R X O5
- CG2: R O6
- Strongest form of experimental control – takes into account re- test effects
Quasi experimental designs
- Time series
- Multiple time series
time series
- When one group is available, and little control over manipulation and participants
- O1 O2 O3 X O4 O5
- Sensitive for history effects and re-test effects
Multiple Time Series
- Time series with control group
- EG: O1 O2 O3 X O4 O5
- CG: O6 O7 O8 O9 O10
- Different tests possible (pre vs post / EG vs CG)
Factorial Design
- Design with more than one IV
* Test for interaction / moderation effects
Full Factorial Experiment
If IV’s are categorical and DV is continuous, as is often the case in experimental designs
One IV
- Two experimental conditions –> T-test, Univariate Analysis of (co-) Variance (One-Way ANOVA)
- Two or more experimental conditions –> Univariate Analysis of (co-) Variance (One- Way ANOVA)
More than one IV
Multivariate Analysis of (co-) Variance (e.g., Two-Way ANOVA)
