detecting test bias

Question 1

what is test bias?

Answer 1

represents various possible discrepancies across subgroups that constitute spurious sources of variance having no intrinsic connection with the construct

Question 2

what does the existence of bias do to validity?

Answer 2

reduces the validity of the interpretation of test scores across one more more subgroups

Question 3

what causes test bias?

Answer 3

due to one or more items in the test that are ‘favouring’ one group over the other

Question 4

what are the objectives of test bias?

Answer 4

• evaluate whether it is present
• try to identify the cause
• make changes to reduce or eliminate it

Question 5

what is a composite score?

Answer 5

a score derived from a lot of other scores

Question 6

what is step one of rank order consistency?

Answer 6

calculate the means associated with each item separately for each group

Question 7

what is step two of rank order consistency?

Answer 7

calculate the spearman rank correlation coefficient between the item means

Question 8

what does a positive correlation when doing a rank order mean?

Answer 8

a positive correlation between the item means equal to or greater than .90 would suggest no construct bias

Question 9

what does a smaller correlation (under .90) suggest?

Answer 9

further investigation needed, possible construct bias

Question 10

what does an eigenvalue tell you?

Answer 10

tells you the percentage of variance the item accounts for

Question 11

what does it mean if an item has an eigenvalue of 2?

Answer 11

that item has 2% of the variance

Question 12

what is congruency coefficient?

Answer 12

an estimation of the correlation between the two factor loadings

Question 13

what are the congruency coefficient cut offs?

Answer 13

.80 - no evidence
.60 - .79 = weak
.30 - .59 =moderately strong

Question 14

do you need to examine statistical significance for the congruency coefficient?

Answer 14

no

Question 15

if there is no predictive bias what will we see?

Answer 15

the degree to which scores from a test predict criteria should be the same across groups

Question 16

what does predictive bias represent?

Answer 16

the systematic differences in a tests capacity to predict various criteria

Question 17

what suggests the presence of predictive bias?

Answer 17

statistically significant differences in intercepts and/or slopes

Question 18

what do overlapping 84% confidence intervals show?

Answer 18

no significant difference so no evidence for predictive bias

Question 19

what do no overlapping confidence intervals show?

Answer 19

there is a significant difference so there is evidence for intercept bias

Question 20

when comparing whether the confidence intervals overlap what bounds do we look at?

Answer 20

upper bound of the smaller estimate and lower bound of the higher estimate