L4: Modern Test Theory

Question 1

What are the advantages of classical test theory?

Answer 1

• intuitive & easy to apply

Question 2

What are the disadvantages of classical test theory?

Answer 2

• focus is on the test not on the items
• test properties depend on the population
• person properties depend on test properties

Question 3

What are the advantages of modern test theory?

Answer 3

• focus on test items
• population independent
• test independent

Question 4

What are the disadvantages of modern test theory?

Answer 4

• statistically not that intuitive
• need larger samples

Question 5

What are the assumptions of modern test theory?

Answer 5

unidimensionality (only measure 1 construct)

Question 6

What is the construct score in classical test theory?

Answer 6

sum score

Question 7

What is the construct score in modern test theory?

Answer 7

score on the latent variable

Question 8

What is the Y axis for IRT?

Answer 8

probability of a correct answer

Question 9

What is the X axis for IRT?

Answer 9

latent variable

Question 10

What are the various IRT measurement models?

Answer 10

1) Rasch Model
2) 2PL
3) 3PL
4) Graded Response Model

Question 11

What are the elements of the Rasch model?

Answer 11

1) Trait Level
2) Item Difficulty

Question 12

What are the elements of the 2 Parameter Logistic Model?

Answer 12

1) Trait Level
2) Item Difficulty
3) Item Discrimination

Question 13

What are the elements of the 3 Parameter Logistic Model?

Answer 13

1) Trait Level
2) Item Difficulty
3) Item Discrimination
4) Guessing Probability

Question 14

What are the elements of the Graded Response Model?

Answer 14

1) Trait Level
2) Item Difficulty
3) Item Discrimination

Question 15

When do you use the Graded Response Model?

Answer 15

When there are more than two response options (not binary)

Question 16

Item characteristic curve - Definition

Answer 16

reflect the probabilities with which ppl across a range of trait lvls are likely to answer each item correctly

Question 17

Item difficulty - defintion

Answer 17

the trait lvl required for participants to have a 0.5 probability of answering the item correctly

Question 18

What feature of the s shaped curve does the item difficulty influence?

Answer 18

where the middle of the curve is on the x axis (max)

Question 19

What feature of the s shaped curve does the item discrimination influence?

Answer 19

determines the slope of the curve (steepness)

Question 20

What feature of the s shaped curve does the guessing probability influence?

Answer 20

determines the lower bound of the curve (y intercept)

Question 21

What is represented by θᵢ?

Answer 21

Trait level

Question 22

What is represented by αᵢ?

Answer 22

Item discrimination

Question 22

What is represented by βᵢ?

Answer 22

Item difficulty

Question 23

What is represented by cᵢ?

Answer 23

guessing probability

Question 24

What are the features of the s shaped curve in a Graded Response Model?

Answer 24

• Each item has a curve for each of the possible response options
• First option → s shaped curve that decreases
• Last option → s shaped curve that increases
• Options in the middle → parabola

Question 25

What are the applications of item response theory?

Answer 25

1) Scale Analysis
2) Test Construction
3) Study Item Fairness
4) Computerised Adaptive

Question 26

Differential Item Functioning - Definition

Answer 26

Differential item functioning occurs when an items properties in one group are different from the item properties in another group

Question 27

Computerized Adaptive Testing - Definition

Answer 27

method of computerized test administration that is intended to provide an accurate & very efficient assessment of individual’s trait lvls

Question 28

How does computerised adaptive testing work?

Answer 28

• Computer selects the item which gives the most info FOR YOU
• Selects items w a probability of a correct response of 50%
• Answer correctly → will then select a more difficult item
• Answer incorrectly → will then select a less difficult item
• Will continue until the estimate for the latent trait ✕ change anymore (becomes stable)