week 13 education bias in measurement Flashcards
static testing vs dynamic testing
Static Testing – providing a single test to assess a
student in a controlled environment
Dynamic Testing – obtaining assessments from a variety of sources including observations of learning
- Involves multiple assessments, including observations of how someone learns or adapts during the test.
- Provides insights into potential and learning processes.
Achievement Test
tests knowledge or skills that someone has learned (example - the SAT)
purpose: evaluates educational progress or mastery of a subject
What is an aptitude test?
- assess the potential to learn
- evaluates unknown and uncontrolled experiences
- based on the ability to predict a future criterion
example: IQ, pre employment tests
what is bias
systematic error in measurement
- occur either how a measured was constructed or how we collection info via sample
Acquiescence Bias
individuals will always agree or disagree without actually considering the meaning of the statement
ex. answering “yes” to all questions
Extreme and Moderating Bias
choosing only extreme options (ex. strongly agree) or middle ground option (ex. neutral)
Social Desirability Bias
most well known response bias
Tendency for people to respond in a way that is socially appearing, regardless of their true characteristics
- not random
example: exaggerating positive traits
two types of social desirability bias
1) Impression Management – respondents consciously
respond to appear more desirable.
* tends to appear when the
consequences of the test are high -> promotions or hiring,
criminal, etc
2) Self-deception – respondents unconsciously hold unrealistic views of themselves and struggle to respond truthfully
- This overestimates their own psychological
characteristics
Malingering Bias
Exaggerating of faking problems, often for person gain
examples:
in disability evaluations or personal injury
examinations there is a pressure to have worse effects in order to ensure the decision you want is made
or
children tend to over exaggerate symptoms, making this bias more present in younger ages
Recall Bias
when an individual has trouble recalling a past
event being asked, resulting in an incorrect response provided
(e.g., forgetting key details when asked about past experiences)
Construct Bias
when the relationship between the observed and true score differs between two groups
- test measures different things for different groups due to cultural or linguistic differences
Predictive Bias
When a test predicts outcomes better for one group than another
e.g., SAT predicting GPA better for men than women
selection bias
individuals have different probabilities of being
included in our validation sample
- done on a skewed population that doesn’t represent everyone
Situation Bias
External factors (e.g., test conditions or examiner behavior) influence results
Test Bias
Systematic errors in the test design favoring some groups over others
- considered the most controversial bias
What is a Raven Progressive Matrix
A non-verbal intelligence test using visual puzzles (patterns and shapes) to measure general intelligence (g-factor)
- can be used as young as 5 years age with no language skills
what are the benefits of Raven Progressive Matrix
- no language required (can be used all across the world)
- culturally fair, with less racial/ethnic bias compared to others
- measures reasoning ability & problem solving skills
What did DuBois argue about static testing?
argued that static testing favours those with more learning opportunities, creating self-fulfilling prophecies
Static tests often fail to assess someone’s potential to improve with support or instruction, leading to biased conclusions about ability
What is ceiling effect
Occurs when a test has a maximum score, and individuals hit this limit, making it impossible to measure higher abilities = dont know their true knowledge
example:
a math test maxes out at 100 - students who get 100 might have abilities above but are then treated equally
why does ceiling effect impact measurement
- Reduces test reliability and validity
- Prevents differentiation between high-performing individuals.
- Can lead to underestimating the true ability of participants.
ethical concerns - labelling
Assigning labels (e.g., “gifted” or “disabled”) can stigmatize and create self-fulfilling prophecies, affecting outcomes
- has a tendency to make subjects stop from
overcoming barriers
Example: being labeled with Schizophrenia is often stigmatizing
ethical concerns - human rights
Subjects have a right to know their
- test scores
- interpretations of
tests
- decision that may impact their lives.
- any test bias.
- who will access the test results
- any confidentiality protections
ethical concerns - privacy
ensuring privacy is not invaded
- Test-takers must know who has access to their results and how data will be used.
- Protect confidentiality at all times
ethical concerns - dehumanization
As we use more technology, a test becomes more
dehumanizing
- we need to be evaluating any risks of removing human interaction to the test procedure
- ensure storage of information is
minimized
