L4: Modern Test Theory Flashcards
What are the advantages of classical test theory?
- intuitive & easy to apply
What are the disadvantages of classical test theory?
- focus is on the test not on the items
- test properties depend on the population
- person properties depend on test properties
What are the advantages of modern test theory?
- focus on test items
- population independent
- test independent
What are the disadvantages of modern test theory?
- statistically not that intuitive
- need larger samples
What are the assumptions of modern test theory?
unidimensionality (only measure 1 construct)
What is the construct score in classical test theory?
sum score
What is the construct score in modern test theory?
score on the latent variable
What is the Y axis for IRT?
probability of a correct answer
What is the X axis for IRT?
latent variable
What are the various IRT measurement models?
1) Rasch Model
2) 2PL
3) 3PL
4) Graded Response Model
What are the elements of the Rasch model?
1) Trait Level
2) Item Difficulty
What are the elements of the 2 Parameter Logistic Model?
1) Trait Level
2) Item Difficulty
3) Item Discrimination
What are the elements of the 3 Parameter Logistic Model?
1) Trait Level
2) Item Difficulty
3) Item Discrimination
4) Guessing Probability
What are the elements of the Graded Response Model?
1) Trait Level
2) Item Difficulty
3) Item Discrimination
When do you use the Graded Response Model?
When there are more than two response options (not binary)
Item characteristic curve - Definition
reflect the probabilities with which ppl across a range of trait lvls are likely to answer each item correctly
Item difficulty - defintion
the trait lvl required for participants to have a 0.5 probability of answering the item correctly
What feature of the s shaped curve does the item difficulty influence?
where the middle of the curve is on the x axis (max)
What feature of the s shaped curve does the item discrimination influence?
determines the slope of the curve (steepness)
What feature of the s shaped curve does the guessing probability influence?
determines the lower bound of the curve (y intercept)
What is represented by θᵢ?
Trait level
What is represented by αᵢ?
Item discrimination
What is represented by βᵢ?
Item difficulty
What is represented by cᵢ?
guessing probability
What are the features of the s shaped curve in a Graded Response Model?
- Each item has a curve for each of the possible response options
- First option → s shaped curve that decreases
- Last option → s shaped curve that increases
- Options in the middle → parabola
What are the applications of item response theory?
1) Scale Analysis
2) Test Construction
3) Study Item Fairness
4) Computerised Adaptive
Differential Item Functioning - Definition
Differential item functioning occurs when an items properties in one group are different from the item properties in another group
Computerized Adaptive Testing - Definition
method of computerized test administration that is intended to provide an accurate & very efficient assessment of individual’s trait lvls
How does computerised adaptive testing work?
- Computer selects the item which gives the most info FOR YOU
- Selects items w a probability of a correct response of 50%
- Answer correctly → will then select a more difficult item
- Answer incorrectly → will then select a less difficult item
- Will continue until the estimate for the latent trait ✕ change anymore (becomes stable)
