Week 5 Flashcards
What is item response theory?
A modern measurement theory that focuses on the relationship between items and the total score or latent dimension underlying the test
More often used in the assessment of ability than personality
Key differences between CTT and IRT
CTT: sample dependent IRT: sample invariant
CTT: scoring is simple
IRT: assumes there is a rship between responses in items and the latent dimension being measured
CTT: relies on representativeness of the sample to generalise to broader population
IRT: has stronger assumptions > unidimensionality (items assess a single construct) > local dependence (items are not too similar)
What is an Item Characteristic Curve (ICC)?
The relationship between the probability of a correct response on a true/false item and the underlying dimension can be assumed to take the form of a cumulative normal distribution.
What is discrimination in an ICC?
The ability to separate/distinguish individuals who are high on a trait to those low on a trait/underlying dimension.
Steeper curve = more discrimination
Flatter curve = less discrimination
What is difficulty or threshold in an ICC?
the curve closer to the left side of the chart is easier, the curve closer to the right side is harder
What is pseduo-guessing?
The third parameter of an ICC can be used to estimate the probability of a response for people with very low levels of an underlying dimension.
It’s guessable
What is differential item functioning?
This questions item bias; whether the item behaves differently for people from two groups who are at the same level of the underlying dimension i.e. depression by gender
How can we remove item bias?
We can change:
the measure
the question (re-word)
This is the strength of IRT - it helps to identify where particular bias in items exists.
To obtain the best result we must rule out measurement effects.
What are the steps of test construction in IRT?
Prepare specifications
Prepare item pool
Field test the items
Select the test items
Compilation of norms, if required
Specify cut-off scores if required
Reliability and validity checks
Final test production
What are IRT parameters?
Discrimination
Difficulty
Pseudo-guessing
Once these are known from a large sample, we can choose items that provide the best estimate of a persons level on a latent dimension
What is CAT logic?
Computerised adaptive testing is a series of rules that are applied to items to determine their appropriateness for a test
Example:
A. 567 + 235 = (difficulty=0)
B. 456/56 =
(difficulty = 1)
C. 24 + 78 =
(difficulty = -1)
D. 10 + 15 =
(difficulty = -1.5)
