Lecture 8

Question 1

What are the three main things Item Response Theory addresses?

Answer 1

1. test bias
2. adaptive testing
3. item selection

Question 2

What is a key skill in professional psychology?

Answer 2

measurement

Question 3

What is Classical Test Theory often know as?

Answer 3

the theory of total scores

Question 4

What is the problem with CTT?

Answer 4

we can only observe the test score, we cannot see depression for example

Question 5

What is central to IRT?

Answer 5

the relationship between the item and the overall construct being assessed- the thing that we cannot really see. It assumes that there is a relationship between responses to items and the underlying/latent dimension assessed by the scale.

Question 6

In CTT the estimates of the test and item parameters are dependent on?

Answer 6

the sample from which they were calculated- so what is the relevance to a clinical sample when the test has been derived from undergraduates.

Question 7

What is an advantage of CTT?

Answer 7

scoring is usually simpler

Question 8

What does an item characteristic curve do?

Answer 8

describes the relationship between the probability of a correct response on a true/false item and the probability of having the underlying dimension

Question 9

What are the numbers on the y axis (probability of responding)?

Answer 9

0 (very unlikely) to 1 (certain)

Question 10

What are the numbers on the x axis (underlying/latent dimension)

Answer 10

anything- we make it up. There are no units for latent dimensions. Usually a mean of 0 and a SD of 1.

Question 11

IRT are sample invariant, what does this mean?

Answer 11

they do not depend on the sample they are drawn- e.g. we can still get useful information clinically if it has only been tested on undergraduates

Question 12

IRT are uni….., what does this mean?

Answer 12

unidimensionality- only assess a single construct.

Question 13

IRT has local… what does this mean?

Answer 13

local dependence- has items that are assessing the same thing but are not the same item, they is just enough difference to be usueful

Question 14

In the Item characteristic curve, what is the slope an estimate of?

Answer 14

discrimination

Question 15

In the Item characteristic curve, the point on the X axis is an estimate of?

Answer 15

the difficulty or threshold

Question 16

In the ICC, if the curve is more to the left or more to the right, what does this mean?

Answer 16

more to right= harder difficulty, better discrimination, more to the left = easier difficulty, less discrimination

Question 17

In the ICC, a steeper slope means?

Answer 17

more discriminating

Question 18

What is a possible third parameter for the ICC (after discrimination + difficulty)

Answer 18

pseudo-guessing- estimates the probability of a response for people with very low levels of the underlying dimension

Question 19

What did Georg Rasch propose was the best way to make a questionnaire?

Answer 19

have items that have similar discrimination that differ in difficulty

Question 20

What does ICC for multiple choice items plot?

Answer 20

a separate curve for each response

Question 21

What is non-parametric IRT?

Answer 21

you are not reading the data with assumptions of what it’s going to say, you are taking it for what it is

Question 22

Discuss the National Survey of Mental Health and Wellbeing

Answer 22

• questioned 7746 people who reported having 12 or more drinks in the past 12 mths.
• were asked about alcohol dependence + abuse
• 1.9% met criteria for alcohol abuse
• 4.9% met criteria for alcohol dependence

Question 23

What information do we get on the alcohol survey if we use Classical Test Theory?

Answer 23

• cronbach alpha (measure of internal consistency)
• note that the relationship b/w an item and the total score is expressed by a single number (item total correlation)
• the closer the ITC gets to 1 the stronger the relationship between the item and the total score

Question 24

What information do we get on the alcohol survey if we use Item Response Theory?

Answer 24

larger, cutdown & tolerance are the most helpful items to decide whether the person has a problem or not
shows that legal problems is not predictive of mental illness

Question 25

Can you use just a subset of the items (tolerance, withdrawal, large cutdown) to give adequate information?

Answer 25

yes

Question 26

Discuss the effect of the number of parameters estimated

Answer 26

1- Rasch model. assumes all items have same slope or discrimination and differ only on difficulty or threshold.
2- include difficulty and discrimination
3- add parameter for pseudo-guessing

Question 27

What is item bias?

Answer 27

looking inside a questionnaire and be able to see if each question is as easy/fair as it could be e.g. do individuals with the same level of depression answer/respond differently to questions re depression?

Question 28

What are 3 ways to fix item bias?

Answer 28

• come out with different scoring methods
• reword items
• decide that it isn’t a good item and not use it anymore

Question 29

What is field testing?

Answer 29

IRT requires larger samples but the randomness of the sample is less important bc IRT is sample invariant

Question 30

What can the selection of test items be on the basis of?

Answer 30

achieving the desired test information function

Question 31

What is item banking and adaptive testing?

Answer 31

once the IRT parameters are known from a large sample it is possible to choose items that provide the best estimate of a persons level on the latent dimension e.g. maybe we don’t have to ask everyone the same questions

Question 32

Where do you start when trying to formulate adaptive testing?

Answer 32

the mean (an item with high discrimination and average difficulty)

Question 33

What is the pyramidal testing model show?

Answer 33

how an individual might pass through a certain number of items (predicts what answers they might choose)

Question 34

what is CAT logic

Answer 34

1. begin with an initial score estimate> select & present optimal scale item > score response > re-estimate health score and confidence interval