detecting test bias Flashcards
what is test bias?
represents various possible discrepancies across subgroups that constitute spurious sources of variance having no intrinsic connection with the construct
what does the existence of bias do to validity?
reduces the validity of the interpretation of test scores across one more more subgroups
what causes test bias?
due to one or more items in the test that are ‘favouring’ one group over the other
what are the objectives of test bias?
- evaluate whether it is present
- try to identify the cause
- make changes to reduce or eliminate it
what is a composite score?
a score derived from a lot of other scores
what is step one of rank order consistency?
calculate the means associated with each item separately for each group
what is step two of rank order consistency?
calculate the spearman rank correlation coefficient between the item means
what does a positive correlation when doing a rank order mean?
a positive correlation between the item means equal to or greater than .90 would suggest no construct bias
what does a smaller correlation (under .90) suggest?
further investigation needed, possible construct bias
what does an eigenvalue tell you?
tells you the percentage of variance the item accounts for
what does it mean if an item has an eigenvalue of 2?
that item has 2% of the variance
what is congruency coefficient?
an estimation of the correlation between the two factor loadings
what are the congruency coefficient cut offs?
.80 - no evidence
.60 - .79 = weak
.30 - .59 =moderately strong
do you need to examine statistical significance for the congruency coefficient?
no
if there is no predictive bias what will we see?
the degree to which scores from a test predict criteria should be the same across groups
what does predictive bias represent?
the systematic differences in a tests capacity to predict various criteria
what suggests the presence of predictive bias?
statistically significant differences in intercepts and/or slopes
what do overlapping 84% confidence intervals show?
no significant difference so no evidence for predictive bias
what do no overlapping confidence intervals show?
there is a significant difference so there is evidence for intercept bias
when comparing whether the confidence intervals overlap what bounds do we look at?
upper bound of the smaller estimate and lower bound of the higher estimate
