IRT

Question 1

IRT’s desirable objectives (2)

Answer 1

(1) Administer SHORTER measures
(2) Compare scores across: DIFF measures of the SAME constructs in DISTINCT groups

Question 2

Why is there a problem in administrating shorter measures according to CTT?

Answer 2

Problem bc relationship between LENGTH of test & RELIABILITY of test
-> Shorter test don’t have as high reliability as longer

Question 3

Limitations of CTT (3)

Answer 3

(1) Adding/deleting items changes true score (because the true score is TEST-DEPENDENT, so comparison not possible across diff test forms)
(2) True score is interpretable ONLY in reference to NORM sample’s distribution of scores: SAMPLE-DEPENDENT
(3) Reliability of true score is function of the items used: All items of EQUALLY reliable, measure SAME RANGE of scores, reliability CONSTANT across scores

Question 4

What’s the problem with CTT assumption that “Reliability of true score is function of the items used”?

Answer 4

In practice, some items are better than some others

Question 5

Item Response Theory (IRT) Assumptions (4)

Answer 5

(1) True score defined on the latent trait dimension rather than observed score
(2) Knowing properties of item a person endorses tell us the trait level the person possesses
(3) Properties of an item do NOT change if we were to administer the item using different samples
(4) True score of the person does NOT change regardless of which sets of items we administer.

Question 6

In IRT, we place both ___________________ and ___________________ on the same scale to be able to compare those two.

Answer 6

items characteristics; people characteristics

Question 7

IRT is a family of mathematical models that describe the probability of a given response to an item as a function of _______________ and ____________. It models the _______________.

Answer 7

certain item characteristics; respondent true score; likelihood of you endorsing an item

Question 8

IRT: What’s the chance you’re gonna answer YES to an item assessing HIGH attachment levels?

Answer 8

KNOWING what’s the level of attachment of an ITEM
+
Underlying level of INDIVIDUAL attachment
= likelihood of you saying yes.

Question 9

Item Response Function

Answer 9

Representation of the probability of item endorsement across the range of true scores
=> models the likelihood of item endorsement across the entire range of underlying traits

Question 10

IRT: TRUE SCORE =

Answer 10

PROB OF ENDORSING ITEMS WITH SPECIFIC CHARACTERISTICS given the trait level set.

Question 11

Item Characteristic Curve (ICC)

Answer 11

Function that models the likelihood of endorsement => plot of the Item response function

Question 12

Item Response Function

Answer 12

Probability that a person with a given ability level will answer CORRECTLY.
=> EQUATION that relates true score (theta) defined in latent dimension to the probability of endorsing an item.
=> DIFF CURVES FOR DIFF ITEMS!!!

Question 13

Variables in Item Response Function

Answer 13

Y = Probability of item endorsement (“yes”) = HOW MUCH TRAIT LEVEL YOU POSSESS
X = Theta (latent trait) - e.g. entire range of math level
Theta is a CONTINUUM (from -infinity to +infinity)

Question 14

Theta def + values

Answer 14

Entire range of latent trait.
=> CONTINUUM (from -infinity to +infinity)
=> Negative values = LOW levels
=> Positive values = HIGH levels

Question 15

How does a typical ICC looks for items that are dichotomous (yes-no)?

Answer 15

S shape

Question 16

Whare are item characteristics?

Answer 16

Item difficulty & Item discrimination

Question 17

What’s the “nature” of the ICC function?

Answer 17

MONOTONIC: Probability of item endorsement increases in theta.

Question 18

ICC: In the middle of the curve, ____ changes in theta correspond with ___ changes in probability

Answer 18

small; large

Question 19

ICC limited by 0 and 1, why?

Answer 19

Bounds of probability: You can never say a probability is ZERO or ONE (impossible); never reaching those two points

Question 20

Item Difficulty def

Answer 20

b
The point in theta (X axis) where probability of endorsing an item is 50%.
=> To find it, start by checking 0.5 in the Y axis
=> Then you find what’s the level of theta (X) that correspond to item difficulty

Question 21

Item difficulty typically range between ______

Answer 21

– 2 and + 2
(-/+ 2 = Arbitrary z-score)

Question 22

Item difficulty:
=> NEGATIVE difficulties = _____
=> POSITIVE difficulties = ______

Answer 22

Items are “EASIER”, more frequently endorsed (doesn’t take much of the trait level to endorse);
Items are more “DIFFICULT”, less frequently endorsed

Question 23

[Item difficulty] What does it mean if Theta > b

Answer 23

Items more likely to be endorsed
=> When theta level is HIGHER than difficulty of the item

Question 24

[Item difficulty] What does it mean if Theta < b

Answer 24

Items less likely to be endorsed
=> When level of underlying trait LOWER than item difficulty

Question 25

Theta = b

Answer 25

= 50%; item difficulty

Question 26

Item Discrimination

Answer 26

a
Value of the slope at the STEEPEST point of the curve, i.e.,b= 50%;
-> Point in the curve where the increases in Y are the highest.
To find it: find theta for difficulty -> this is the point where beta is the most elevated
=> The steeper the line, the closer it is to vertical.

Question 27

Item Discrimination is related to ______

Answer 27

Item difficulty

Question 28

Item Discrimination tells us ________

Answer 28

at which levels of data the item is most likely to differentiate best
=> Discriminates levels of theta

Question 29

Discrimination typically ranges between _____

Answer 29

.5 and 1.5

Question 30

[Item discrimination] What does it mean when…
=> Steeper slopes
=> Smaller slopes

Answer 30

Highly discriminating items;
Poorly discriminating items

Question 31

Items would be most effective in measuring underlying trait at the level that correspond with _______.

Answer 31

item difficulty
→ Hard questions are more effective at measuring high levels of the trait.

Question 32

Item Information Curve indicates _________

Answer 32

How well an item is working for EACH LEVEL of the trait.
= How well an item differentiate among respondents who are at different levels of the latent variable
= Item difficulty + Item discrimination

Question 33

Item parameters determine the amount of information at what range of the latent trait. (1) What are the parameters (2) What info do they give

Answer 33

Item difficulty → Location on the latent trait where information is MAXIMIZED
Item discrimination → HOW MUCH INFO an item provides

Question 34

Test Information Curve indicates _______

Answer 34

How much information is the TEST measuring? How well this is test at measuring things with precision?
=> The relative precision of the test in measuring diff levels of the data.

Question 35

When talking about Test Information Curve (TIC), we’re talking about Validity or Reliability? Why?

Answer 35

We’re talking about RELIABILITY (NOT VALIDITY)
Bc it focuses on how precisely a test measures the latent trait ACROSS DIFF LEVELS OF THAT TRAIT.
=> *THE HIGHER THE CURVE, THE BETTER YOUR ASSESSMENT OF THE TRAIT (mountain)

Question 36

The height of the TIC is (inversely) proportional to the ____________

Answer 36

Standard error of measurement (SEM)
=> TIC and SEM are inversely related.
-> Relibility = how much test scores are free of measurement error

Question 37

SEM is ____ in regions of latent trait continuum where test information is the _____.

Answer 37

lowest; highest

Question 38

In IRT, SEM is different for different latent trait values; how is that different from CTT?

Answer 38

CTT: 1 score of reliability for entire set of items
IRT: 1 item = 1 reliability coefficient; Measurement error is NOT equal across the entire range of data

Question 39

How does IRT Help us Improve Psychological Tests? (4)

Answer 39

(1) IDENTIFY item characteristics (i.e., difficulty, discrimination)
(2) CHOOSE items with higher discrimination covering the entire range of the latent continuum
(3) INCREASE RELIABILITY with fewer items
(3) COMPARE itemps across DIFF MEASURES of SAME CONSTRUCT + Compare group differences

Question 40

IRT Applications (2)

Answer 40

(1) Improving existing measures
(2) Detecting differential item functioning

Question 41

Differential Item Functioning (DIF) examines ______

Answer 41

Whether scales and items function differently across different discrete groups.
-> Occurs when groups (such as defined by gender, ethnicity, age, or education) have different probabilities of endorsing a given item (controlling for overall score)

Question 42

Differential Item Functioning (DIF) occurs when _________________

Answer 42

individuals from diff groups who have EQUAL levels of the UNDERLYING TRAIT, have diff probabilities of endorsing or agreeing with an item.

Question 43

DIF analysis helps determine if items are ____ by _____________.

Answer 43

fair; examining group differences in responses while controlling for the trait level