Discovering statistics Flashcards
what does SPINE of statistics stand fro
standard error parameter interval estimates null hypothesis testing estimation
general linear model
outcome = b0 + b1(predictor) + error
chi sqaured test
chisq.test(data$variable, data$variable, correct = FALSE)
for categorical or count data
spearman correlation
data %>% correlation::correlation(., method = “spearman”)
continuous data
what do the parts of GLM stand for
b0 = estimate value when predictor=0 b1 = represents difference in means if linear model has two categorical groups bn = estimate of parameter for predictor, direction/strength of effect, difference of means
least squared estimation
- when no predictors, we predict them outcome from intercept
- outcome = b0 + e
- b0 will be mean value of outcome in this scenario
- if given data estimate the mean
- rearrange equation -> error = outcome - b0
- square the error and plot
- keep estimating mean
- peak of graph is least squared error
standard error
- frequency distribution -> plot sample mean against frequency
- smaller sampling distribution means smaller SD but is called standard error
central limit theorm
majority of scores around mean
normal distribution
1.96 sd from mean contains 95% data
confidence intervals
express estimates as intervals such that we know population value lies in them
95% chance contains true pop parameter
interpreting parameter estimates
raw effect size is the beta estimate
standardised effect size fits model to raw data that are z -scores (expressed in standarised scores)
long run probability: parameters represent effects
relationships between variables
differences in means
long run probability: parameters reflect hypotheses
h0 : b = 0, b1 = b2
h1 : b =/= 0, b =/= b2
long run probability: test statistic
t= b/SEb
can work out how likely value if null true
value of t on x axis and probability on y
type 1 error
reject null when it is true
believe in effects that dont exist
type 2
accept null when its false
statistical power
probability of test avoiding type 2 error
problems with null hypothesis testing
- not tell importance of effect
- little evidence about null hypoth
- encourages all or nothing
- based on long run probability
problem with long run probability
p is relative frequency of observed test statistic relative to all test statistics from infinite no. of identical experiments with exact same priori sample size
type 1 error rate either 0 or 1
comparing sum of sqaures
- sum of squares represent total error
- only compare the totals when based on same number of scores
illusory truth effect
repetition increases perceived truthfulness
equally true for plausible and implausible statement
SSt
-total variability between mean and scores
-SSm + SSr
-each SSt has associated df
dfT = N-p (p = parameter, N = independent information)
SSr
- total residual/error variability
- error in model
- to get SSr we estimate using ‘two’ parameters
- dfR = N - P, so P is 2
SSm
- total model variability
- improvement due to model
- model rotation of null model
- null and estimated model are distinguished by b1
- dfM = dfT - dfR
mean squared error
- sum/total amount of squared errors depends on amount of information use to compute it
- can’t compare sums as based on different amounts of info
- MSr = SSr/df (average residual error)
- MSm = SSm/df (average model variability)
F statistic
- testing fit
- sig fit represents sig effect of experimental manipulation
- if model results in better prediction than the mean then MSm > MSr
- Anova(model_lm)
testing the model
- R^2 proportion of variance accounted for by model
- pearson correlation coefficient between observed and predicted scores^2
- R^2 = SSm/SSr
- adjusted R^2 estimate of R^2 in population
broom: :glance(data_lm)
how to enter predictors
- hierarchal (experimenter decides)
- forced entry (all entered simultaneously)
- stepwise (only used for exploratory analysis, predictors selected using semi partial correlation with outcome)
influential case
- outliers distort linear model and estimations of beta values
- detect them in: graphs, standardised residual, cooks distance, DF beta statistics
- ggplot::autoplot(data_lm, which = 4, …) + theme_minimal() gives estimate, std.error, p.value and removes outliers
robust estimation
- use of model as can’t remove outliers
- robust::lmRob(outcome~predictor, data = data)
- summary(lm_rob)
key assumptions of linear model
linearity (relationship between predictor and outcome is linear) and additivity (combined effects of predictors)
spherical errors (pop model have homoscedastic errors and independent errors)
normality of errors
errors vs residuals
- model errors refer to differences between predicted values and observed values of outcome variable in POP model
- residuals refer to differences between predicted values and observed of outcome in SAMPLE model
spherical errors
- should be independent
- pop error in prediction for one case should not be related to error in prediction for another case
- errors should be homoscedastic
- violation of assumption
homoscedasticity of errors
variance of pop errors should be consistent at different values of predicted variable
violation of assumption
b’s unbiased but not optimal
standard error incorrect
robust procedures
boostrap -> standard errors derived empircally using resampling technique, designed for small samples, robust b, p, and ci
heteroskedasticity -> consistent SE, uses HC3 or HC4 methods
dummy coding
- code control group with 0 and the other with 1
- b for dummy variable is difference between means of two conditions
- mean condition 1 = b0 + b1(0)
- mean condition 2 - mean condition 1 = b1
- dummy coding isn’t independent as used same p-value
contrast coding model
- outcome = b0 + b1(contrast 1) + b2(contrast 2)
- b0 is value of control
- b1 is difference between b1 and b0
- b2 is difference between b2 and b0
planned contrasts
variability explained by model, SSm, due to participants being assigned to diff groups
variability represents experimental manipulation
what to consider when choosing contrasts
- independent - to control for error 1, if group is singled out in contrast then it shouldn’t be used again
- only contrast 2 chunks of variation
- k-1, end up with one less contrast than no. groups
- first contrast compare control to all experimental ones
rules of coding planned contrasts
1-groups coded with positive weights compared to groups coded negatively
2-sum of weights equal 0
3-if group not used code it as 0
4-initial weight assigned is equal to number of groups in opposite chunk
5-final weight = inital/no. groups with non 0 weight
post hoc tests
in absence of hypothesis compare all means
inflates type 1 error rate
use bonferroni to correct
modelbased::estimate_contrasts(data_lm. adjust = “bonferroni”)
trend analysis
polynomial contrast
only ordered groups
contrast(data$predictor)
comparing means
- when know extraneous/confounding variable influences outcome so adjust for them
- reduce error variance by explaining some of unexplained variance
- gain greater insight into effects of predictor
partitioning variance
total variance = explained by predictor + unexplained variance
unexplained variance overlapped by variance explained by predictor and covariate
how do you get beta estimates
broom::tidy(data_lm, conf.int = TRUE)
adjusting means using predicted b values from broom::tidy
covariate and contrasts
- use dummy coding and mean of covariate
- outcome = b0 + b1(contrast 1) + b2(contrast 2) + b3(covariate)
- outcome = 1.7 + 2.2(contrast 1) + 1.7(contrast 2) + 0.4(covariate)
- code contrast 1 as 0, and covariate as its mean, outcome = 2.9
- repeat with contrast 2 coded as 0
unadjusted model
no covariate
predicted values are raw group means
beta attached to contrast 1 is difference between means of individual conditions from within that contrast
f statistic with multiple predictors
calculated for sums of squares
type1: default in R, each predictor evaluated taking into account previous predictors, order of predictor matters
type 3: each predictor evaluated taking into account all other predictors, order not matter
code for type 3 sums
data_lm %>% car::Anova(., type = 3)
bias in f-statistics: heterogeneity of regression
- for sig of f-stat to be accurate we assume relationship between outcome and covariate is similar across groups
- known as homogeneity
- when assumption is met, f stat is assumed to follow f distribution and corresponding pvalue
factorial design
2 or more predictors have been manipulated
moderator
acts on relationship between predictor and outcome
outcome = b0 + b1(predictor) + b2(moderator) + b3(predictor x moderator)
interaction term
predictor x moderator
if interaction term pvalue is significant you ignore all other rows
it its significant there is significant moderator effect
parameter estimate quantifies raw effect size of interaction term
in factorial designs, effect of moderator is stronger is certain categories of predictor
fitting the model factorial design
afex::aov_4(outcome ~ predictor*moderator + (1|id), data = data) doesn’t show parameter estimates, diagnostic plots or robust methods, but afex_plot() plots the interaction
why does pvalue not tell us anythign abou timportance?
it depends upon sample size
what is value range for fstat
0-1, anything greater than 1 means the model explains more than it doesn’t
what test is never used for normality
K-S test
what does the assumption of normality primarily apply to?
sampling distribution of parameters
what is an outlier
data point that is unrepresentative of relationship being investigated
what do the f stat and its associated pvalue tell you
- the ratio between variance explained by model and residual variance
- whether the model explains variance in outcome better than the grand mean
- likelihood of obtaining the value you have if no true difference in means of groups
characteristics of orthogonal contrasts
hypothesis driven
control type 1 error rate
planned a priori
how does f ratio chnage when using dummy, contrast or post hoc test
it doesnt as looks at model as whole
why do you use an interaction term in model?
expecting the effect of one predictor to vary as function of another predictor
what is orthogonal
independent contrasts that cross multiply = 0 and add together = 0
what is main effect
effect of just one of the independent variables on dependent variable
effect f predictor alone ignoring all other predictors in model
assumption of sphericity
automatically met when variable has only two levels
if not met it is remedied by adjusting degrees of freedom by the degree to which data are not spherical
interpret effects of interaction term
a. The extent to which the type of variable A affected outcome depended on type of variable B and vice versa
types of variance
systematic-created by our manipulation
unsystematic-created by unknown factors
benefits of repeated measure design
more sensitive -unsystematic variance reduced, more sensitive to experiemntal effects
more economic-less participants
possibel fatigue effects
repeated measures and linear model
all participants in all conditions, scores correlate
violates assumption of independent residuals
need to adjust model to estimate this dependency:
outcome = boj + bj(predictor) + ej
boj = bo +uoj
b1j = b1 + u1j
u is variability across different particpants
approaches to repeated measure and GLM
assume sphericity: estimate and correct for it
fit multigrowth model
what is sphericity
difference between pairs of groups should have equal variance
assumption the variances are the same between conditions
greenhouse geisser estimate
e=1 then perfect sphericity
what to do for sphericity
r multiples df by e to correct for effect of psherciity
given that e quantifies deviation from perfect spherciity
df get smaller which makes harder to tests tat to be sign
routinely apply g-g correction
repeated measure linear model
afex::aov_4(outcome ~ predictor + (predictor|id), data = data)
ges is effect size
how to set contrasts for self using afex and emmeans
emmeans::emmeans(model_afx), ~predictor, model = “multivariate”)
data_cons
robust model of repeated measures
WRS2::rmanova(y = data$predictor, groups = data$predictor. blocks = data$id)
gives f stat, df
use WRS2::rmmcp for robust post hoc
simple effects analysis
emmeans::joint_tests(data_afx, “predictor b”)
effect of predictor a within predictor B
post hoc test for repeated measures
pairs(int_emm, adjust = “holm”)
don’t do if get non-significant
mixed design contrasts
categorical predictors must be coded as contrast variables
extract them using emmeans::contrasts()
what are the group of methods that can be used in a mixed design
eff -> each category compared to average of all categories
pairwise -> each category compared to all others
poly -> polynomial contrasts
trt.vs.crtl -> compares each category to a declared reference category, ref = x
consec -> compares each level/category to the previous
code to look at main effect
only important if interaction term is non-significant
emmeans::emmeans(data_Afx, ~predictor, model = “multivariate”)
look at each predictor separately: (data_afx, c(“predictor”,”predictor”), model = “multivariate”)
how to adjust for sphericity
R multiples df by value of epsilon, which makes result mroe conservative
what percent of variance in ‘festivity’ is explained by ‘film’
look at column ges
0.15 ges is 15%
what element of model has largest effect size?
largest value of F
why can’t categorical outcomes be a normal linear model?
violates assumption of linearity
model for predicting probability of outcome
ln(P(Y)/1-P(Y)) = b0 +b1(X) + e
outcome is log odds of outcome occurring
b1 is change in log odds of outcome associated with unit change in predictor
log and exponents
log of 1 = 0
exponent of 0 = 1
odds ratio: b0
log odds of outcome when predictor is 0
easier to interpret tahn exp(B0)
odds ratio: b1
- change in log odds of outcome associated with unit change in predictor
- easier to interpret exp(b1), odds ratio associated with unit change in predictor
- OR >1; as predictor increases probability of outcome increases
- OR <1; as predictor increases, probability of outcome decreases
classification table
states number of type of ‘presents’ and how may were ‘delivered’ and ‘undelivered’
odds(delivery)
number of delivered/number undelivered
odds(delivered after treat1)
number of delivered after treat1/ number of undelivered treat2
odds ratio
odds(delivered after treat2) / odds(delivered after treat1)
how to interpret an odds ratio of 0.15
odds of delivery is much smaller for treat 2 than treat 1
0.15 times smaller
fitting GLM to categorical outcome
glm(outcome ~ predictor, data = data, family = binomial())
data_glm %>% parameters::parameteres() %>% parameters::parameters_table(p_digits = 3)
how can you convert the log odds of glm to exponentials
insert “exponentiate = TRUE” into parameters::parameters()
two predictors in categorical outcome
ln(P(Y)/1-P(Y)) = b0 + b1(pred1) + b2(pred2) + b3(pred1 x pred2) + e
things that can wrong with categorical outcome model
linearity spherical residuals multicollinearity incomplete information complete separation
what is incomplete information
empty cells
inflates standard errors
problem escalates quickly with continuous predictors
what is complete separation
outcome variable can be perfectly predicted
how are log odds produced when from a glm
- if produces a log odd of -1.03
- create glm with subsetting the type of treat
- 1.03 = log odd of treat 2 - log odd of treat 1
planned contrasts and parameter estimates
-control_vs_exp
