35 Test Bias Flashcards
What does bias refer to in a test?
Bias in a test refers to construct-irrelevant components that result in systematically lower or higher scores for identifiable groups of examinees.
What is the result of bias for two individuals of same ability?
In a biased test, they score differently if they belong to different subgroups.
What is an example of a construct-irrelevant component?
A written test of building site safety might be biased against non-English speakers. Verbal comprehension is a construct-irrelevant component here: the test is designed to measure knowledge of safety principles.
Does bias have to be systematic?
Yes, it has to be systematically higher or lower for identifiable sub-groups.
Does basing a reading comprehension task on a book that only some participants have read create bias?
No, bias has to be towards or against identifiable sub-groups.
Against whom could depression item “I often feel weary” be biased against?
Elderly people, those with disabilities – who are tired but not depressed. HADS - version of DAS for hospitals, that doesn’t contain physical symptoms.
Are group differences in test scores an indication of test bias?
Not necessarily. The test may have accurately reflected group differences in ability. e.g., girls are dumb
How can difference in prediction be evidence of bias?
When test predicts outcome more strongly for one subgroup than another. For example, if ATAR predicts uni marks better for private than state schools.
What is differential item functioning (DIF)?
A mathematical procedure that tests for bias at an item level. DIF occurs when individuals (or subgroups) with the same level of the attribute obtain different scores on an item or subset of a test.
If the regression line is different for different genders on a test designed to predict performance, what does that mean?
Test may be biased against one of the genders, given that same score on a test predicts different outcomes.
