Lecture 6 - Test Bias Flashcards
What is bias?
-a bias refers to the presence of a systematic difference in the estimation of scores on a given construct (=validity) for a given group.
How is bias linked to margin of error?
-evey test contains a margin or error
-a bias is present when this margin of error differs systematically from one group to another
Is bias the same thing as mean difference?
-NO, when 1 group scores, on average, lower than another, this does not represent a bias
-groups may truly differ from one another on the construct that is being assessed, even if the causes of these differences are unknown
What are the types of bias?
-content validity bias [when an item is systematically more difficult in one group than another]
-construct validity bias [when you’re not measuring the same thing in the two groups]
-predictive validity bias [when you’re not predicting the outcomes with the same precision and quality in the two groups
What is content validity bias?
-an item or sub-scale is deemed biased when it is systematically more difficult for members of one group compared to another
How do we test content validity bias?
1) create subsamples with comparable scores on the test within each cultural groups.
2) within these subsamples , assess the item difficulty level to see if it differs across cultural groups.
(just using expert judges to “assess” whether the item content is proper doesn’t work… they are members of specific cultures)
What is construct validity bias?
-when the test assesses different things in different groups or measures identical things with different degrees of precision
How do we measure construct validity bias?
-there is no one way to assess this
-its about looking at the accumulation of validity evidence about a construct/test (e.g., factor analysis)
What is predictive validity bias (in relation to a criterion)?
-occurs when the quality/precision of the prediction differs across groups
[some argue this is the most important bias]
-does the test allow us to predict, with the same precision and accuracy, each individuals’ performance?
What are the two types of prediction validity bias?
-slope bias [prediction is impossible to compare]
-intercept bias [bias can be corrected if you know that it’s there]
What is slope bias?
-when the coefficient of validity is not the same across groups (need to be careful to take into consideration the range of scores, the sample size, etc.).
-“THE” most severe form of bias. [test cannot be used]
What is intercept bias?
-when the intercept differs across groups. i.e. an equivalent score on the test systematically predicts a higher level of performance in one group relative to the other. This type of bias, once identified, is easy to adjust.
-same score on the test, predicts a higher level of performance in one group than another; but that prediction/difference between group is the same across all possible scores on the test
What is a possible solution for intercept bias?
-adding a constant to the score of everyone in the disadvantaged group would solve this
