Experimental Research Methodology Flashcards
Epistemology
Epistemology
Deduction: derive the special from the general
Inductions: Get of the special on the general
Create hypothesis –> Get data –> structure data –> learn
Deductive logic –> Analysis –> Interpretation –> Inductive logic
Experimental analysis
Experiments (primary data)
Use only deductive hypothesis for the design of experimental research
The data can support or refute The hypothesis, or can build up new inductive theories.
ANOVA
Real data (secondary data)
Real data can be analyzed With deductive as well as inductive ideas
Linear regression
Treatment:
Treatment: experimental manipulation
Factor:
Factor: experimental manipulated variable (Independent variable)
Factor level:
Factor level: discrete expression of the factor
experimental cell:
experimental cell: experimental group within the design
Control group:
Control group: experimental group without treatment
Between subjects design:
Between subjects design: each sub this is assigned to only one exactly one experimental cell
Within subjects design:
Within subjects design: every subject is assigned to every level of the experimental factor
Mixed factorial design:
Mixed factorial design: combination of between and within design
2x2 design
Paper: to test our hypothesis, we employed other 2 (brand name: business versus relaxed) x 2 (font type: Low complexity versus high complexity) between subjects design.
Between subjects design
Random assignment of subjects to groups
Measure –> treatment/control –> measure
Advantages: All scores are independent and not influenced by earlier treatments
Disadvantages: Differences in groups, different environments, compensatory behavior
Stratified random assignment
Random assignment of subjects under a certain variable (f.e.: 25% male + 75% female)
Within subjects design
Every subject gets MULTIPLE treatments
Measure –> Treatment 1 –> Measure –> Treatment 2 –> Measure
Advantages: No individual differences (each participant is it’s own control) + less participants required
Disadvantage: Practice/Carry over effects
Experimental procedure
Step 1: deductive derivation of the hypothesis
Derive hypothesis from a series of argumentation.
Step 2: operationalization of the independent variable
take a factor and vary it (For example factor a contrast)
Manipulate a strong as possible, but don’t overdo it.
Step 3: operationalization of the dependent variable
Use some kind of measurement: subjective rating, objective behavior, neurophysiologic activities
Step 4: operationalization of the control or interference variable Measure these variables to reduce the error variance Personality variable (Stable over time): intelligence or Joy of learning Situational variable (Unstable overtime): affective mood or concentration
Step 5: Between versus within subjects design
Between subjects: every person Gets assigned exactly to one experimental condition. The other group does not know, that there is a second treatment
within subjects: every person is assigned to all experimental conditions sequentially (in random order).
It is important that the subjects cannot conclude onto the reason for the experiment.
Step 6: determination of the number of probands:
How big should the number of subjects D, that we can see an effect with an acceptable probability (convention: 80 %).
This sample size depends on 4 criteria: (Most of the time N= 128 for between subjects & N = 34 for within subjects)
Tolerated size of the Alpha error (Convention: 5%)
Tolerated size of the beta error (Convention: 20%)
Number of experimental groups
Forecasted/estimated size of the effect.
Multi factorial design
Multi factorial designs: Interaction effect (A influences B if C)
Interaction/moderation: 2 effects work together multiplicatively, that The impact the result more than the simple sum.
Two hypothesis: Hypothesis one versus hypothesis two (2 x 2 Experiment between versus within)
Questions for evaluation of an experiment
Questions:
Is there a main effect of the factor “cognitive control”?
Is there a main effect of the factor “contrast”?
Is there an interaction of the Factors? (Is the effect of the factor contrast depending on the factor cognitive control?)
Follow up analysis: is there an effect of the factor of contrast with higher cognitive control?
Follow up analysis: is there an effect of the factor of contrasts with Low cognitive control?
ANOVA
Analysis of variance: ANOVA
Checks whether there are a significant mean differences between different groups (independent variable, UV) for the one dependent variable (AV)
F test on significance: sets systematic variance (AV, you can describe through UV) in relation to Error variance.
Internal validity
Internal validity: every systematic variation in the dependent variable is clearly on the manipulation of the independent variable attributable. Causal interputation only works, if there is no plausible Alternative.
potential risks to internal validity: systematical distortions in the collection of data; non-random assignment of subjects; cofoundation in the operationalization of the experimental manipulation.
Pitfalls of experimental research
Internal validity & external validity
External validity
External validity: the results of a study are generalizable on other operationalizations, times,Places and different subjects. A causal interputation only works, if it’s generalizable.
Potential dangers for external validity: artificial experimental setup; unrealistic experimental stimuli; unrepresentative sample (only students).
Mediator
Definition: the mediator ME is a variable, which connects an independent variable X with an dependent variable Y causally. The statistic concept of mediation is conceptionally based on the SOR Paradigm
