03_Experimental data analysis Flashcards
What is the problem of inflation of type-1-error in experimantl data?
Type-1-error: rejecting the null hypothesis when it is actually true
->Type I error inflation means that there is an increased risk of mistakenly rejecting a null hypothesis
If we test 4 different commercials with dependent variable x:
- prob of type-1-error in 1-test: 0.95 (1-alpha)
- prob of type-1-error in 6-test: (1-alpha)^6 –>0.74
Thus, we require a procedure that compares cell means simultaneously
–>Analysis of variance
What are the central assumptions of using ANOVA?
▪ Random allocation to groups
– Structural equality of the experimental groups regarding potential confounding variables
– Can be checked by evaluating the randomization process & by control variables
▪ Data is normally distributed
– Observations within the groups are approximately normally distributed
(especially relevant if the average group size ≤ 15)
– Can be checked using histograms or suitable statistical tests
– If applicable: remove outliers
▪ Equal group size
– Experimental groups are approximately of equal size
– If not met: randomly remove cases or apply adjustments for the ANOVA
▪ Similar variance in all experimental conditions
– The variance of the dependent variable is approximately the same in all experimental conditions
– Formal check through corresponding tests possible (however: Type I error inflation)
– Rule of thumb: The largest standard deviation should at most be twice the size of the smallest standard deviation
What does ANCOVA account for ?
Elimination of rival causal explanations (confounding variables) often fails:
- Randomization not entirely sucessful: systematic influence of omitted variables
- Problem of small sample sizes
–>Causal interpretation of the ANOVA dependent on accounting for all confounding variables
–>Solution: ANCOVA accounts for the Covariance of the dependent variable with the potential confounding variable (covariate)
What are the two forms of impact off the confounding variable in analyiss of ANCOVA?
1.Correlation between Y and Z within groups only
2.Correlation between Y and Z between the groups only
—>ANCOVA
What is an interaction effect?
Interaction effect
- the impact of one factor is also dependent on values of other factors
- interaction effets point to moderation effects
What do interaction effects have im common with moderators?
interaction effects point to moderation effects
moderator: moderation effect occurs when the impact of one variable on another is contingent upon the level of a third variable. This third variable is referred to as the moderator.
What are the two types of interactions?
No interaction vs. interaction effect:
- If the slope does not change, it is just a level shift–> no interaction effect
Disordinal vs. ordinal interaction:
- disordinal: order changes moving i.e. from high pressure to low pressue (in effect it increase, in the other one it decrease)
- Ordinal: order changes are the same by moving from high to low
cross-over interaction vs. no cross-over interaction:
The two variables cross each other, only possible in disordinal
What does the analysis of variance test only?
Analysis of variance only test the effects of the independent variables in total
–>problem: frequently we want to know, where do specific differences between the attributes of the independent variables exits?
Solution: contrasts test and bonferonni correction
What are the two techniques used, if we want to know specifc differences between attributes ofthe IV exists?
1.Contrasts tests: already before research specific hypothesis concering the differences between single groups are fomulated
- If you have a significant result from ANOVA/ANCOVA, then use contrasts test
2.Bonferroni correction/Post hoc tests –>specific hypothesis according to the differences between single groups were not formulated
What is the bonferroni correction?
Starting issue: by having several groups, we hav a type-1-error inflation problem –>that´s why we use ANOVA
Alternative: stricter requirements for the allowed significane level
Bonferroni correction: adaption of significance level:
- new significance level
alpha´ =< (a/m)
a= actual accepted significance level (5%)
m= number of test to be conducted
What does the ANOVA do?
ANOVA uses amount of explained variation and relates it to the amount of unexplained variation [errors]
What is ANCOVA?
ANCOVA is an extension of ANOVA that includes one or more continuous covariates in addition to the categorical independent variable (factor).
–>ANCOVA is used when you want to compare group means while statistically controlling for the effects of one or more covariates.
