Stats and research Flashcards
idiographic
single subject
nomothetic
group studies
AB design problem
threat of history - hard to tie intervention to change
ABAB design problems
protects against threat of history
threat of failure to return to baseline
ethics of stopping intervention
analogue research vs. clinical trial
less generalizability vs methodological compromise
cross sectional design problem
cohort effect
longitudinal problems
expensive, attrition rate
cross-sequential research
combine longitudinal and cross sectional
proportional sampling
randomly selected in proportion to representation
systemic sampling
every nth random
cluster sampling
naturally occurring groups, but everyone within the groups
Threats to internal validity
anything other than IV causing change in DV (history, maturation, test practice, instrumentation)
Solomon four-group design
Corrects for test practice
pre/post vs. post only
intervention vs. no intervention
Instrumentation
threat to internal validity
change in observer or equipment resulting in change to DV. Correct with control group
Statistical regression/ regression to the mean
extreme scores get less extreme
manage with control group
selection bias
correct with random assignment
attrition/experimental mortality
problem if differential in group dropout rates
Compare groups who drop out with t tests
Diffusion
no-treatment group gets some treatment indirectly
Experimental expectations/ Rosenthal effect
Experimenter transmit clues. correct with experimenter blindness
Demand characteristics
features of procedures that suggest expectations to participants
John Henry effect/compensatory rivalry
control group work harder to compete with experimental group. Try to keep control and experimental group unaware of each other.
Threat to external validity
sample characteristics, stimulus characteriastics, contextual characteristics
Sample characteristics
diff between sample and pop
Stimulus characcteristics
features of study associated with intervention
Contextual characteristics
conditions in which intervention embedded. Reactive/Hawthorn effect
Threats to statistical conclusion validity
low power, unreliability, variability in procefures,. subject heternogeity.
positive skew
peak to left
neg skew
peak to right
standard error of mean
all possible means from random assignment groupings, how far they deviate from middle
Central limit theorem
Rejection region / region of unlikely value
area corresponds to
Type 1 error
mistakenly rejecting Ho
proportionate to alpha
Type 2 error
mistakenly adopt Ho
Inverse with alpha
Choosing based on DV
non-parametric if nominal/ordinal
parametric - t-test, anova
1+ DV - MANOVA
independent vs correlated data
independent if randomly assigned, or based on existing categories (e.g., gender)
Correlated if same-subject repeated measures, matched to groups, inherent relationships between ppl in each group
parametric test assumptions
interval or ratio data
homoscedascity - -similar variance between groups
normally distributed data
chi square test assumptions
independent observations - no repeated measures
Degrees of freedom
possible variations in outcome
Chi square: groups -1
Multiple chi square: (rows-1)*(columns-1)
Single sample t-test: N-1
Matched/correlated t-test: #of pairs -1
T-test for independent groups: N-2
single ANOVA: N-1, #of groups -1,
post-hoc tests
scheffe, then tukey are best for protection from type 1
Fisher’s LSD best for proection from type 2
vice versa
two-way anova benefit
allows analysis of interaction effects
F ratios for each IV and each interaction
interaction effects must be examined first
MANOVA vs multiple ANOVAs
reduce type 1 error
coefficient of determination
correlation coefficient squared, explains variability in Y accounted for by X
Simple regression line of best fit
uses least squares criterion
assumptions for bivariate correlations
linear relationship between x and y, homoscedascity, unlimited ranges for x and y
eta
use when X and Y relationship is curvilinear
Multicollinearity
multiple predictors in multiple regression are highly correlated with each other/redundant
stepwise regression
computer generated
forward - add one predictor at a time until no change to r2
backward - remove one predictor at a time, weakest first, until there is change to r2
Fewest possible predictions
higerarchical regression
examiner adds predictors by hand according to theoretical model
Canonical R
correlation between 2+ IVs and 2+ DVs (predictor set and criterion set)
Discriminant function analysis
2+ IV, 1 nominal DV
Loglinear/Logit analysis
2+ categorical IV, 1 categorical DV
Path analysis
Causality
multiple regressions to test causality model
structural equation modeling
inferences about casality
e.g., LISREL (linear structural relations)
Factor analysis
reduce several variables into fewer factors
1st factor always strongest
eigenvalue/characteristic root
factor strength
1+ is significant
correlation matrix
table of intercorrelations among tests/items
orthogonal rotation
produces factors that have no correlations with each other, easy to interpret
communality - factor loadings squared and added together
oblique rotations
factors are correlated like in real world
principal components analysis
no empirical/theoretical guide on communality values
produces uncorrelated factors called components
1st factor accounts for most var
principal factor analysis
communality value confirmed before analysis
Cluster analysis
look for naturally occuring groups of DVs
No a priori hypothesis
minimum acceptable reliability is
0.8
Spearman-Brown prophesy formula
tells how much more reliable test would be if it was longer
on timed tests, preferred reliability measure is
alternate tests, then test-retest
power test
items of varying difficulty
Kuder-Richardson
split-half reliability for dichotomous repsonse formats
KR 20 when varying difficulty
KR 21 when equal difficulty
error due to content sampling, test heterogeneity
Cronbach’s
Interater reliability
Pearson’s r, % agreement, Yule’s Y, kappa statistic
standard error of measurement
SD of normal distribution from tested 100s of times theoretically
reliability
standard error of estimate
validity
SD of normal distribution from being tested 100 of times theoretically on criterion
Item characteristic curve
Rel between item score and total score
Item response theory
to what extent does specific item correlate with underlying construct
cross validation
give test to new sample
shrinkage of criterion-related validity coefficient
correction for attenuation
calculates how much more valid it would be if predictor and criterion always reliable
criterion contamination
happens for subjective criterions, rater informed of subject’s predictor scores before assigning criterion ratings
ANCOVA
extra/moderator variables partialled out
Looking for group diff, not factors
Spearman’s rho/kendall tau
corr between ordinal variables
pearson r
interval/ratio
point biserial
corr between true dichotomy and interval/ratio
biserial
corr between artificial dichotomy and interval/ratio
phi
corr between two dichotomies
tetrachoric
corr between two artificial dichotomies
Taylor Russel tables
Moderate base rate, low selection ratio, incremental validity
slope bias
occur when there is differential validity - i.e., when the validity coefficients for a predictor (e.g., cognitive ability test), differ for different groups. Consequently, the predictor is more accurate for one group than for another.
intercept bias
Intercept bias (or unfairness) occurs when the validity coefficients and criterion performance for different groups are the same, but their mean scores on the predictor differ. As a result, the predictor consistently over- or under-predicts performance on the criterion for members of one of the groups.
