IRT Flashcards

1
Q

IRT’s desirable objectives (2)

A

(1) Administer SHORTER measures
(2) Compare scores across: DIFF measures of the SAME constructs in DISTINCT groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Why is there a problem in administrating shorter measures according to CTT?

A

Problem bc relationship between LENGTH of test & RELIABILITY of test
-> Shorter test don’t have as high reliability as longer

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Limitations of CTT (3)

A

(1) Adding/deleting items changes true score (because the true score is TEST-DEPENDENT, so comparison not possible across diff test forms)
(2) True score is interpretable ONLY in reference to NORM sample’s distribution of scores: SAMPLE-DEPENDENT
(3) Reliability of true score is function of the items used: All items of EQUALLY reliable, measure SAME RANGE of scores, reliability CONSTANT across scores

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What’s the problem with CTT assumption that “Reliability of true score is function of the items used”?

A

In practice, some items are better than some others

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Item Response Theory (IRT) Assumptions (4)

A

(1) True score defined on the LATENT trait dimension rather than observed score
(2) Knowing **PROPERTIES OF ITEM **a person endorses tell us the TRAIT LEVEL the person possesses
(3) Properties of an item do NOT change if we were to administer the item using different samples
(4) True score of the person does NOT change regardless of which sets of items we administer.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

In IRT, we place both _______ and _______ on the same scale to be able to compare those two.

A

items characteristics; people characteristics

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

IRT is a family of mathematical models that describe the probability of a given response to an item as a function of _______________ and ____________. It models the _______________.

A

certain item characteristics; respondent true score; likelihood of you endorsing an item

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

IRT: What’s the chance you’re gonna answer YES to an item assessing HIGH attachment levels?

A

KNOWING what’s the level of attachment of an ITEM
+
Underlying level of INDIVIDUAL attachment
= likelihood of you saying yes.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Item Response Function

A

Representation of the probability of item endorsement across the range of true scores
=> models the likelihood of item endorsement across the entire range of underlying traits

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

IRT: TRUE SCORE =

A

PROB OF ENDORSING ITEMS WITH SPECIFIC CHARACTERISTICS given the trait level set.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Item Characteristic Curve (ICC)

A

Function that models the likelihood of endorsement => plot of the Item response function

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Item Response Function

A

Probability that a person with a given ability level will answer CORRECTLY.
=> EQUATION that relates true score (theta) defined in latent dimension to the probability of endorsing an item.
=> DIFF CURVES FOR DIFF ITEMS!!!

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Variables in Item Response Function

A

Y = Probability of item endorsement (“yes”) = HOW MUCH TRAIT LEVEL YOU POSSESS
X = Theta (latent trait) - e.g. entire range of math level
Theta is a CONTINUUM (from -infinity to +infinity)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Theta def + values

A

Entire range of latent trait.
=> CONTINUUM (from -infinity to +infinity)
=> Negative values = LOW levels
=> Positive values = HIGH levels

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

How does a typical ICC looks for items that are dichotomous (yes-no)?

A

S shape

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Whare are item characteristics?

A

Item DIFFICULTY & Item DISCRIMINATION

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What’s the “nature” of the ICC function?

A

MONOTONIC: Probability of item endorsement increases in theta.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

ICC: In the middle of the curve, ____ changes in theta correspond with ___ changes in probability

A

small; large

19
Q

ICC limited by 0 and 1, why?

A

Bounds of probability: You can never say a probability is ZERO or ONE (impossible); never reaching those two points

20
Q

Item Difficulty def

A

b
The point in theta (X axis) where probability of endorsing an item is 50%.
=> To find it, start by checking 0.5 in the Y axis
=> Then you find what’s the level of theta (X) that correspond to item difficulty

21
Q

Item difficulty typically range between ______

A

– 2 and + 2
(-/+ 2 = Arbitrary z-score)

22
Q

Item difficulty:
=> NEGATIVE difficulties = _____
=> POSITIVE difficulties = ______

A

Items are “EASIER”, more frequently endorsed (doesn’t take much of the trait level to endorse);
Items are more “DIFFICULT”, less frequently endorsed

23
Q

Item difficulty: What does it mean if Theta > b

A

Items more likely to be endorsed
=> When theta level is HIGHER than difficulty of the item

24
Q

Item difficulty: What does it mean if Theta < b

A

Items less likely to be endorsed
=> When level of underlying trait LOWER than item difficulty

25
Q

Theta = b

A

= 50%; item difficulty

26
Q

Item Discrimination

A

a
Value of the slope at the STEEPEST point of the curve, i.e.,b= 50%;
-> Point in the curve where the increases in Y are the highest.
To find it: find theta for difficulty -> this is the point where beta is the most elevated
=> The steeper the line, the closer it is to VERTICAL.

27
Q

Item Discrimination is related to ______

A

Item difficulty

28
Q

Item Discrimination tells us ________

A

at which levels of data the item is most likely to differentiate best
=> Discriminates levels of theta

29
Q

Discrimination typically ranges between _____

A

.5 and 1.5

30
Q

[Item discrimination] What does it mean when…
=> Steeper slopes
=> Smaller slopes

A

Highly discriminating items;
Poorly discriminating items

31
Q

Items would be most effective in measuring underlying trait at the level that correspond with _______.

A

item difficulty
→ Hard questions are more effective at measuring high levels of the trait.

32
Q

Item Information Curve indicates _________

A

How well an item is working for EACH LEVEL of the trait.
= How well an item differentiate among respondents who are at different levels of the latent variable
= Item difficulty + Item discrimination

33
Q

Item parameters determine the amount of information at what range of the latent trait. (1) What are the parameters (2) What info do they give

A

Item difficulty → Location on the latent trait where information is MAXIMIZED
Item discrimination → HOW MUCH INFO an item provides

34
Q

Test Information Curve indicates _______

A

How much information is the TEST measuring? How well this is test at measuring things with precision?
=> The relative precision of the test in measuring diff levels of the data.

35
Q

When talking about Test Information Curve (TIC), we’re talking about Validity or Reliability? Why?

A

We’re talking about RELIABILITY (NOT VALIDITY)
Bc it focuses on how precisely a test measures the latent trait ACROSS DIFF LEVELS OF THAT TRAIT.
=> *THE HIGHER THE CURVE, THE BETTER YOUR ASSESSMENT OF THE TRAIT (mountain)

36
Q

The height of the TIC is (inversely) proportional to the ____________

A

Standard error of measurement (SEM)
=> TIC and SEM are inversely related.
-> Relibility = how much test scores are free of measurement error

37
Q

SEM is ____ in regions of latent trait continuum where test information is the _____.

A

lowest; highest

38
Q

In IRT, SEM is different for different latent trait values; how is that different from CTT?

A

CTT: 1 score of reliability for entire set of items
IRT: 1 item = 1 reliability coefficient; Measurement error is NOT equal across the entire range of data

39
Q

How does IRT Help us Improve Psychological Tests? (4)

A

(1) IDENTIFY item characteristics (i.e., difficulty, discrimination)
(2) CHOOSE items with higher discrimination covering the entire range of the latent continuum
(3) INCREASE RELIABILITY with fewer items
(3) COMPARE itemps across DIFF MEASURES of SAME CONSTRUCT + Compare group differences

40
Q

IRT Applications (2)

A

(1) Improving existing measures
(2) Detecting differential item functioning

41
Q

Differential Item Functioning (DIF) examines ______

A

Whether scales and items function differently across different discrete groups.
-> Occurs when groups (such as defined by gender, ethnicity, age, or education) have different probabilities of endorsing a given item (controlling for overall score)

42
Q

Differential Item Functioning (DIF) occurs when _________________

A

individuals from diff groups who have EQUAL levels of the UNDERLYING TRAIT, have diff probabilities of endorsing or agreeing with an item.

43
Q

DIF analysis helps determine if items are ____ by _____________.

A

fair; examining group differences in responses while controlling for the trait level