Study Guide 10: Latent Ability, Item Characteristics, & Responses: Item Response Theory

Question 1

Binary item

Answer 1

an item with only two possible response options

Question 2

Differential item functioning (DIF)

Answer 2

refers to the case when the ICCs are not the same for the two groups. Specifically, DIF occurs when examinees from different groups show differing probabilities of responding correctly to (or endorsing) an item after being matched on the underlying ability (latent variable).

Question 3

Item response theory (IRT)

Answer 3

• a measurement theory that relates characteristics of items (called item parameters) and characteristics of individuals (called latent traits or latent ability and designed as ‘theta’) to the probability of giving a positive or correct response. IRT is popular because it provides a theoretical justification for doing lots of things that CTT does not.

Question 4

Inflection point

Answer 4

An inflection point is a point on a curve at which the sign of the curvature (i.e., the concavity) changes.

Question 5

Item Analysis:

Answer 5

A set of statistical techniques to examine the performance of individual items. This is important when developing a test or when adopting a known measure.

Question 6

Item bias

Answer 6

an item is biased when knowledge or skills that are not relevant to the construct of interest are needed to either endorse the item or get it right
Item bias occurs when examinees of one group are less likely to answer an item correctly (or endorse an item) than examinees of another group because of some characteristic of the test item or testing situation that is not relevant to the test purpose. DIF is required, but not sufficient, for item bias.

Question 7

Item characteristic curve (ICC)

Answer 7

line showing the relationship between the characteristics of individuals (i.e., the latent trait) and the probability of giving a positive or correct response.

Question 8

Item impact

Answer 8

describes the situation in which examinees from different groups (e.g., male, female) have differing probabilities of responding correctly to (or endorsing) an item because there are true differences between the groups in their underlying ability (on the latent trait).

Question 9

Item impact and item bias differ

Answer 9

in terms of whether group differences are based on relevant or irrelevant characteristics (respectively) of the test. DIF requires that members of the two groups be matched on the relevant underlying ability before determining whether members of the two groups differ in their probability for success.

Question 10

Item information function (IIF)

Answer 10

The three item parameters can be combined into an item information function (IIF). The IIF describes how well, or precisely, an item measures at each level of the latent ability (i.e., theta). Higher information scores are better.

Question 11

Measurement model

Answer 11

expresses the mathematical links between an outcome (e.g., a reponsdent’s score on a particular item) and the components that affect the outcome (e.g., qualities of the respondent and/or qualities of the item)
1PL, 2PL, 3PL models are all examples of the models.
They reflect the idea that an individual’s response to an item is determined by the individual’s trait level and by item properties (eg. Difficulty, discrimination).

Question 12

Non-uniform DIF

Answer 12

the ICCs for the two groups do cross. In non-uniform DIF, which group performs better changes across the latent variable.

Question 13

parameter

Answer 13

The variables a, b, and c, are the parameters of the curve. They vary from item to item and they define the specific shape of the ICC

Question 14

Person fit

Answer 14

IRT is used to estimate item characteristics and then identify individuals whose responses to items do not adhere to those parameters. The analysis of person fit is an attempt to identify individuals whose response pattern does not seem to fit any of the expected patterns of responses to a set of items. Ex. It would be odd to find an individual who endorses a difficult item but doesn’t endorse an easy item.
Implications: poor person fit could indicate cheating, random responding, low motivation, cultural bias of the test, intentional misrepresentation, scoring/admin errors.
In personality assessment: may reveal that their personality is unique.

Question 15

Polytomous item

Answer 15

has more than two response options. These require different IRT models – ex. Graded response model, partial credit model.

Question 16

Rasch model

Answer 16

1PL model. The IRT model in which only the b parameter varies

Question 17

Test information function (TIF)

Answer 17

The test information function (TIF) shows the degree of precision (or quality) in estimating parameters along the latent variable continuum for the entire test. It represents an average of the IIFs for all of the items. Higher information scores are better = more precision. The higher the item’s discrimination the more peaked the curve will be

Question 18

Theta or Person Parameter

Answer 18

Latent ability. In a model, theta is presented as a z-score – the average person’s score is set at 0, and each value represents a standard deviation from the average ability.

Question 19

Uniform DIF

Answer 19

where the ICCs for the two groups do not cross.