Week 8- Item Response Theory

Question 1

What is Item Response Theory?

Answer 1

New theory with a focus on test items that adds more tools for solving measurement problems in psych in relation to test bias, adaptive testing, item selection
It is concerned about the relaitonship between observed responses to items and the underlying dimension or construct.

Question 2

What is the most important thing for a clinical psychologist?

Answer 2

Measurement

Question 3

What is the difference between classical test theory and irt?

Answer 3

CTT focuses more on the total score of a
scale or subscale
• IRT focuses on the relationship between
items and the total score or latent dimension
underlying the test.

Question 4

What is central in IRT?

Answer 4

The relationship between the item and the overall construct.

Question 5

What is a negtive of IRT

Answer 5

because the scoring is more technical it takes longer

Question 6

What does IRT assume

Answer 6

Assumes there is a relationship between responses to items and the underlying or latent dimension being assessed by the scale.

Question 7

What is the item characteristic curve?

Answer 7

The relationship
between the
probability of a correct
response on a true/
false item and the
underlying dimension
can be assumed to
take the form of a
cumulative normal
distribution

Question 8

What is sample invariant?

Answer 8

Something that doesnt depend on the sample they are drawn from

Question 9

What does the slope in the curve represent?

Answer 9

The estimate of discrimination

Question 10

What does the point on the x axis represent?

Answer 10

The difficulty or threshold

Question 11

How do we measure on the item characteristic curve?

Answer 11

Finding where the 50% point of probability of responding is and then dropping down

Question 12

What is a pseudo-guessing parameter?

Answer 12

can be used to estimate the probability of a response for people with very low levels of the underlying dimension

Question 13

Rasch model

Answer 13

Items had same discrimination but that differ in difficulty.

Question 14

National Survey of Mental Health and Wellbeing

Answer 14

•Larger, Cutdown & Tolerance discriminate at close to the estimated diagnostic threshold
•Other criteria are markers of
more severe states
•Estimated diagnostic threshold shown as vertical line

Question 15

What are the two types of IRT models?

Answer 15

Parametric or Non-Parametric.

Question 16

How many parameters are estimated?

Answer 16

1- Parameter or Rasch Models. They assume all items have the same slope or discrimination and differ only on difficulty or threshold. Infrequently used because real life doesnt meet up to the standard.
2 parameter models- like seen in slides (two parameters are discrimination and threshold)
3 parameter models- add a parameter for pseudo-guessing (more widely used in ability testing)

Question 17

What is item bias?

Answer 17

When you look at the question and ask if it is as fair as it could be

Question 18

What a the test for item bias?

Answer 18

Does the item behave differently for people from two groups who are at the same level of the underlying dimension or trait?

Question 19

How do you select test items?

Answer 19

On the basis of achieving the desired test information function e.g. are you testing if someone has a severe alcohol problem or if something may develop an alcohol problem in the future

Question 20

What is item banking and adaptive testing

Answer 20

Choosing items that provide the best estimate a persons level on the latent dimension with known precision

Question 21

How do you start item banking?

Answer 21

You pick at item with a mean that has high discrimination and average difficulty

Question 22

How do you choose the next item

Answer 22

Is chosen to maximise the informatuin (minimize the error of measurement)

Question 23

What is cat item selection?

Answer 23

For example a 4 item bank for a maths test
– Item A “567 + 235 = ?” difficulty =0
– Item B “456 / 56 = ?” difficulty=1
– Item C “24 + 78 = ?” difficulty= -1
– Item D “10 + 15 = ?” difficulty=-1.5
• If Item A correct then the next choice would be item B
and if correct maths ability would be at or above 1 (in
standardised scores mean=0, SD=1)
• If item A incorrect then next choice C then if C
incorrect then item D if D incorrect then ability would
be less than -1.5 (standardised)
• More items would give greater allowance for getting a
single item wrong (increase reliability and reduce
standard error)

Question 24

What are the important rules in Embretson’s new rules of measurement?

Answer 24

Standard error of measurement, assessment of item properties, mixing item formats

Question 25

What is standard error of measurement

Answer 25

New Rule: The standard error of estimation differs across scores but generalises across populations. SE(0)= 1 to the square root of I(0).
The standard error of estimation is the inverse of test information function
If something has more information it has less error

Question 26

What is the assessment of item properties

Answer 26

• Old: Unbiased assessment of item properties
depends on having representative samples
• New: Unbiased assessment of item
properties may be obtained from unrepresentative
samples

Question 27

What is mixing item formats

Answer 27

Old: Mixed item formats leads to unbalanced
impact on test total scores
• New: Mixed item formats can yield optimal test
scores