exam 3 Flashcards
two-way ANOVA
we have two factors, we are interested in the effects of both factors on the same dependent variable, can have ANOVA with 3 or 4 factors
conditions matrix
if we have 2 factors and each factor has 2 levels, we have 4 possible combos of factors, we call this a 2X2 factorial design
three mean comparison
in a 2 factor design, we can test 3 hypotheses all at one: 1) does IV1 have an effect on DV? 2) does IV2 have an effect on DV? 3) does IV1 and IV2 interact to affect the DV?
main effects
first F tests the main effect of factor A as if factor B wasn’t there, second F tests the main effect of factor B as if factor A wasn’t there
interaction
the third F tests for an interaction, based on whether there is any more variance between groups that we haven’t already accounted for
writing about interactions: what to report in two-way ANOVA
same that you report in one-way ANOVA; means, F statistic, df, p-value for overall F, effect size (eta squared)
two-way tables of means
report the means for each cell ( combo of two factors) and rows/columns means
- put levels of factor A in rows and levels of factor B in columns
- use extra columns to left and extra column on top to include factor names
- put SD for each condition in parentheses after the mean
two-way ANOVA table
expand ANOVA table to include SS, df, MS, F and p for each of three tests
- divide into “between and “within” first then break down between variables
- add “*” to help reader quickly see which p values are significant
Writing about two-way ANOVA table
- report the results of each F test separately, but be clear about which effect you’re talking about
- when you have a significant interaction, you need to explain the form of the interaction
- then refer reader to tables and figures for further details
connecting your hypotheses: two-way ANOVA table
as you report your results, link them to hypotheses so reader can keep track of whether you were right or not
conventions for plotting interactions
- an interaction involves at least 3 variables
- DV/outcome always goes on vertical axis
- IV/predictor goes on horizontal axis
- in ANOVA, IV predictor is categorical
- other IV/predictor variable is represented by different lines
how to interpret interaction plots
- do lines cross? if yes; disordinal interaction, if no; ordinal interaction
- are lines separated? if yes, consistent with main effect of IV 2
- do the lines slope? if lines slope up or down thats consistent with main effect of IV1
correlational research
- comes in many forms (direct observation, surveys, analysis of existing data),
- involves more stats than a simple correlation
correlation vs t-test and ANOVA
t-tests and ANOVA are used to compare means of dif groups
when should you use correlation?
useful
- any time predictor variable is continuous
- any time you have at least 1 continuous variable and no strong expectations about which one predicts the other
- correlations are symmetrical, it doesn’t matter which variable comes first
when to use one-sample t-test?
when we know or can assume the pop mean but we don’t know the pop SD
independent samples t-test
comparing two samples with different people in each sample
-also called a between participants design
basic conceptual formula for any t-test
t= difference between means / estimated SE
standard error
error in one sample= SM1= square root of S1^2/n1
pooled standard error
if two samples are not the same size, we need to account for that difference
- pooled variance= Sp^2= SS+S2/df1+df2
- we use pooled variance instead of sample variance to calculate the SE
the pearson correlation
- indicated by the lower-case r (for sample, p for pop)
- r= degree to which X and Y vary together/ degree to which X and Y vary separately
- more shared variability= strong relationship= high correlation
calculating correlation
-we are interested in whether the deviations for one variable are similar to the deviations for the other
simple linear regression
-one predictor= one outcome
things we need to accomplish in a regression analysis
- find a straight line that best fits the data (Y=bX+a), what are best values for b and a?
- evaluate how well that line fits the data
interpreting coefficients (shape) in simple linear regression
b is our regression coefficient; the slope of our regression line
- tells us how strong the relationship is between our predictor and our outcome
- large/small b= change in the predictor corresponds to a large/small change in the outcome
- positive/negative b- an increase in the predictor corresponds to an increase/decrease in the outcome
- a, the intercept, tells us what value we’d expect for the outcome if the value of the predictor was zero
effect sizes for regression
- we can calculate a standardized regression coefficient by transforming all of the raw scores to z-scores before we begin the analysis
- fancy b
- we also can find r^2 for our regression model
multiple regression
-a regression analysis involving more than one predictor variable
overall effect size for multiple regression
- R^2= the amount of variance in the outcome
- so a larger R^2 means we’ve done a better job explaining/predicting the outcome variable
standard error of estimate
- average error of prediction
- tells us how accurate our predictions are when we predict based on all the variables in the model
writing about regression; what to report
- include results of significance test: F stat, df ( regression, residual)
- coefficients in the regression equation
ANCOVA
- combines regression and ANOVA
- test the effect of grouping variables after accounting for continuous variables
- logic: “if i already suspect that Y is probably related to C, does X add anything to my understanding of Y beyond what I get from C?”
ANCOVA process
- start with regression model predicting SV from covariates
- use ANOVA to understand residual variance
interpreting ANCOVA
in ANCOVA, we get F statistics for:
-each main effect
-each covariate
-the interaction (if we have two factors)
we compare them to all appropriate critical values and/or compare the exact p-value for each to our a level
writing about ANCOVA: what to report
when writing about ANCOVA, include results of all F tests
- F stat
- df (regression, residual OR between, within)
- p-value for overall F
effect size and shape: ANCOVA
to interpret shape, look at similar stats
-for your factors: means
-for your covariate; correlation and regression with the DV
for effect size, you can find partial eta squared for each effect
discussion section
- what you found
- what you think it means
- why you think it matters
- what should they keep in ind when interpreting results
- how your findings might be of practical value
- what next steps are for future research
contributions
before you dissect what could have been better about your study, talk about why it has value as is: why was this study worth doing, what does the field gain from this study?
strengths
reasons reader should have confidence in your study
-internal and external validity
limitations
-boundary conditions
-possible limitations internal validity: being unable to control important individual differences, realizing after the fact that you had confounding variables, having too little stat. power or too much noise in data
possible limitations external validity; having unrepresentative sample, having measures too far removed from real world, having manipulations that are too weak or don’t work
future directions
help reader see what comes next, what is next logical step in addressing your research question?
post hoc analyses
- extra analysis to rule out alternative explanations
- NOT part of hypotheses
- NOT same as post hoc that follows ANOVA
why do post hoc analyses?
-helps you address “what if’s”
risk of post hoc analyses
- may not have exactly the data you want
- risk of capitalizing on chance
moderating variables
- moderation occurs when one variable changes the effect of another variable on an outcome
- same thing as an interaction!
testing moderation
- look for interactions
- remember there are three variables in an interaction or moderating relationship
- moderator and the IV/predictor do NOT influence one another
challenges in findings moderators
- moderating effects are too small
- choosing which variable is the focal variable vs the moderator is sometimes messy
- interactions can be difficult to interpret
mediating variables
- if moderation if about when one variable affects another, mediation is about HOW
- “if variable M is casually situated between X and Y and accounts for their association ( at least in part), we can say that M mediates the relationship
- think “media”= middle
how mediation works
a mediating variable explains how we get from one variable to another
-“because…”
-“due to…”
-“by…”
like links in a chain, if we break the link from X to M, then Y doesn’t happen
testing mediation
use path models or structural equations models
-this allows us to test the relationships among all of our variables at the same time
path models and structural equations models
- both are ways to visualize and test a set of hypotheses among multiple variables at the same time
- solve a set of simultaneous regression equations
path models
- deal with observed variables- the data just as we collected it
- we represent the observed variables with boxes
- represent relationship with arrows (one-headed arrow= variable on the left causes or influences the variable on the right, two-headed= variables are correlated, but we don’t know direction)
Structural equations model (SEM)
- just like path models, but with latent variables rather than observed variables (observed= exactly what we saw/counted, latent= something we infer from the data)
- we use latent variables when we aggregate items together to form an overall score
SEM diagrams
-we use ovals to represent latent variables (author used a multi-item measure)
evaluating a path or structural model
- does the overall model fit the data well?
- interpret path coefficients (just like regression coefficients, they describe the strength of each path (arrow) in the middle)
meta-analysis
-you systematically review existing literature and then quantitatively combine the results
advantages of meta-analysis
- they can settle controversy
- more likely to incorporate broader range of studies
- allow us to get a more accurate effect
- can also test for moderators
concerns about meta-analysis
- still not perfectly objective
- for some questions, we get neat, consistent relationships across studies
reading a meta-analysis
- what effect size are they using (cohens d, r^2, and eta squared are all popular)
- pay attention to CI’s as well as effect size estimate
- number of studies (k) is more important the number of participants (n)
eta squared
% of variance in the DV that is explained by one or more IVs in the context of ANOVA
eta squared
tells us what proportion of the variance in our outcome is explained, or predicted, by our grouping variable
r^2
proportion of variance in one continuous variable that is explained or predicted by ONE other continuous variable
R^2
proportion of variable in one continuous variable that is explained or predicted by a SET of continuous variables
sampling error
the natural discrepancy, or amount of error, between a sample statistic and its corresponding population parameter
central limit theorem
for any population with true mean and SD, the distribution of the sample means for sample size n will have a mean of true mean and SD/ sqare root of N and will approach a normal distribution as n approaches infinity
law of large numbers
the larger the sample size (n), the more probable it is that the sample mean will be close to the population mean
standard error
average sampling error
sampling error
the difference between one sample mean and the population mean
interpreting z-scores
pay attention to
- sign (positive=above mean, zero=mean, negative= below mean)
- value- close or far from zero?
p-value
precise probability of a statistic as extreme or more extreme than what we have
type 1 error
we are concluding there is an effect when there really is no effect; reject null when null is really true
type 2 error
we are concluding there is no effect when really there is an effect; accept null when null is really false
avoiding type 2 error
start in design state
- pilot test your manipulations and measures
- think carefully about study administration and potential confounds
- recruit adequate sample
power
the probability that we will correctly reject the null when the null is really false; the likelihood that we find an effect that is really there
traditional framework for inference tests
- state hypothesis
- choose a level
- based on a, find critical region and critical value for test stat
- compute test stat
- compare test stat to critical value
modern framework for inference testing
w/modern software, we can easily find exact probability the we would obtain our exact test stat if null were true
- state hypothesis
- choose a level
- compute test stat ( find exact probability of test you obtained)
- compare exact p-value to a level (if p<a></a>
t distribution
-the set of t-scores for all possible samples of a particular size (n)
three variations on t-tests
-mean of one sample with a known pop mean (one sample)
mean of one sample with a mean of a different sample (independent sample)
-mean of one sample and mean of sample that is matched or connected to the first in some way (paired or dependent samples)
t
difference between means/ estimated SE
one tailed test
- if we make a directional prediction, we’re only right if the effect goes in the direction we predicted
- if we’re interested in only one end of the distribution
two-tailed test
if we’re interested in values at either end of the distribution
one-tailed test and critical region
- critical region is all at the same end of the distribution
- we will only reject H0 when our sample mean is dif. from pop mean in the direction we predicted
- has more power than two-tailed
two-tailed test and critical region
- critical region is divided between two ends
- we can reject H0 with a smaller difference than the mean
- has less power than one-tailed
when would you use a one sample t-test?
when we know or can assume the population mean but we don’t know the population SD
dependent samples t-test
-we have to account for lack of independence
designs with dependent samples
- repeated measures (or within-participants)
- matched participants (or matched pairs)
confidence interval
a range of values centered around the sample statistic, the width of the interval gives us a certain degree of confidence that the pop parameter is inside that interval
writing about t-tests: what to include
- alternative hypothesis in conceptual terms
- what kind of t-test you’re using
- alpha level you’re using and whether you conducted a one or two-tailed test
- df
- exact t stat you calculated
- p value for t stat
- reject or don’t reject H0
- effect size (cohen’s d)
ANOVA
takes same logic as t-tests and extends it so we can compare two or more means
ANOVA; between-treatments variance
-differences between group means
ANOVA; within-treatments variance
-normal variability, regardless of grouping variable
F
systematic group differences + random, unsystematic variability/ random, unsystematic variability
one-way ANOVA
ANOVA with just one factor (grouping variable)
eta squared
percentage of variance in the DV that is accounted for by the IV
eta squared= ssbetween/sstotal
multiple comparisons problem
- when the overall F stat is significant, we know that at least 2 of the means are different
- but we don’t know yet whether only the extremes are different or if there are also differences among the means in the middle
omnibus test
tests the equality of several means all at the same time
post hoc tests
when we test all possible pairs of means for differences
-differences we didn’t specifically predict
Tukey’s honestly significant difference
- square root of MSwithin/n
- only works for equal sample sizes
- easier to calculate than sheffe
Sheffe test
- calculates and separates MSbetween for every pair of groups
- to get a significant F with Sheffe, the difference between 2 groups has to be large enough to get an F that exceeds our critical value for the whole study
- more conservative than Tukey (sets a strict standard for type 1 error)
writing about ANOVA: what to report
- F statistic
- DF (between and within)
- P-value for overall F
- effect size (eta squared)
writing about post hoc tests
- only report these if overall F is significant
- be clear which post hoc test you used
- having a clear, organized table of means allows your reader to make sense of this at a glance
three F tests: two way ANOVA
- one for main effect of factor A
- one for main effect of factor B
- one for interaction
interactions
- occurs when the effect of one IV/predictor variable on the DV changes depending on the level of another IV/predictor variable
- “it depends” findings
calculating for repeated measures
in a repeated measures design, we substitute MSerror for MSwithin in the F statistic
F=between/groups variance/within groups variance= MSbetween/MSerror
effect size
a measure of effect size is intended to provide a measurement of the absolute magnitude of a treatment effect, independent of the size of the sample being used
effect size
tells us how big the effect is
statistical significance
tells us that we can be confident that the effect we see- whatever size- is real
-when we reject the null, we describe our findings as statistically significant- this means that it is unlikely we would have observed our result if the null hypothesis were true
