Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Psychometrics > Item Response Theory > Flashcards

Item Response Theory Flashcards

1
Q

How does IRT modeling differ from Rasch modeling?

A
  • IRT modeling seeks to fit a model to the data (model fitness) whereas Rasch modeling seeks to fit data to a model (model parsimony).
  • Data that does not fit a Rasch model is discarded.
  • Rasch modeling does not permit abilities to be estimated for extreme items and persons
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What are the ____ advantages of IRT over CTT?

A

fdsfd

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What is the purpose of IRT?

A

To relate item characteristics (item parameters) to individual characteristics (latent traits) to create a model that is used to predict the probability of a particular response to an item.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Please list some of the limitations of CTT.

A
  • It is impossible to separate items and examinees
  • *insert here*
  • You are not able to predict the likelihood of a correct response to an individual item
  • You are unable to solve many important practical testing problems
    • Test fairness analysis
    • Computerized adaptive testing
    • Equating test scores
    • Designing tests
    • Dealing with missing data
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What does theta (Θ) represent in IRT?

A

It represents the amount of a latent trait that an individual possesses.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What are two ways of describing what P(Θ) represents?

A
  1. It is the probability of answering item *j *correctly for a randomly sampled examinee with Θ.
  2. It is also the frequency of correct answers for a given examinee with Θ after administering item j repeatedly.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are three other ways to depict P(Θ)?

A
  1. Expected score conditional on Θ (only for dichotomous items)
  2. Regression of score on ability Θ (only for dichotomous items)
  3. P(Θ) is a probability and also a conditional expectation.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

What are the three steps that must be taken when creating an adaptive test?

A
  1. Where should examinees begin (e.g., at what difficulty level should the first items be set at)?
  2. What should we them give next (e.g., after getting the first set of items correct/incorrect, how difficult should the next set of items be)?
  3. When should we have them stop (e.g., how many item sets do we have them complete before we are satsified)?
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What is the probability of a “correct response” to a dichotomous item?

A

P(Θ)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What is the probability of an “incorrect response” to a dichotomous item?

A

1 - P(Θ) = Q(Θ)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is a polytomous item?

A

An item that has a different score for each possible response option.

E.g., Likert scale items or political affiliation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is a dichotomous item?

A

An item that has only two possible score values (regardless of how many response options are available)

E.g., Correct vs. Incorrect

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is the purpose of an item response function (IRF)/item category response function (ICRF)?

A

To graphically represent (a) the probability of an examinee answering an item correctly, or (b) the probability of an examinee endorsing a particular response category.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Why can the IRT assumption of unidimensionality never be truly met?

A

It can never be met because there are always other latent traits that are responsible for a given response (e.g., motivation, reading ability, etc.) for any testing instrument.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is another name for the assumption of local independence?

A

Assumption of conditional independence.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

If the assumption of unidimensionality is true, is the assumption of local independence also true?

A

Yes!

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

If the assumption of local independence is true, is the assumption of unidimensionality true?

A

Not necessarily. For example, a model that include multiple latent traits could account for the complete latent space (satisfying the assumption of local independence) but would not satisfy the assumption of unidimensionality.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What does the item parameter *b *represent?

A

*b *= an item’s location/difficulty

Location = the amount of the latent trait that is needed to have a 0.5 probability of endorsing the item.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

What does the item parameter a represent?

A

a = an item’s discrimination/slope

Discrimination = how strongly related the item is to the latent trait like loadings in a factor analysis

It also is proportional to the slope of the ICC at the point bi (location) on the ability scale (which is the maximum slope of the IRF)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What does a c item parameter represent?

A

c = guessing

Guessing = is included when respondents very low on a trait may still choose the correct answer

This is mostly used with multiple choice testing and should not vary excessively from the reciprocal of the # of response options (e.g., 0.25 would be expected if there are 4 response options).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

What does a *d *item parameter represent?

A

d = upper asymptote

Upper asymptote = included when respondents very high on the latent trait are not guaranteed (i.e., have less than 1 probability) to endorse the item.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
22
Q

What is the typical range of *b *values?

A

-3 to +3

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
23
Q

If a b parameter is adjusted (made bigger or smaller), while holding all other parameters constant, which direction does it move ICC?

A

If adjusted to be smaller then it moves the ICC left.
If adjusted to be larger then it moves the ICC right.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
24
Q

Are the terms item response function (IRF) and item characteristic curve (ICC) interchangeable?

A

Yes!

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
25
Q

How is the assumption of unidimensionality met?

A

It is adequately met when a set of test data contains a “dominant” component or factor that influences the final score.

26
Q

Which of the common dichotomous models is the most general?

A

The three-parameter logistic model (3PLM). This model consists of parameters a (discrimination), b (location/difficulty), and c (guessing).

27
Q

When is the assumption of local independence met?

A

If examinees’ reponses to any pair of items are statistically independent after holding the abilities influencing test performance constant (e.g., one or more latent traits).

28
Q

Are higher *b *parameters associated with test items that are more difficult?

A

Yes!

29
Q

What does a lower *b *parameter represent?

A

It represents an item which requires a lower amount of the latent ability to have a 50% chance of getting the item right.

30
Q

What does the D factor represent?

A

It represents a scaling factor that is introduced to make the logistic function as close as possible to the normal ogive function.

31
Q

What value of D is used in two- and three-paramter logistic models?

A

D = 1.7

32
Q

Is the Rasch model mathematically equivalent to the one-parameter logistic model?

A

Yes!

33
Q

What does the one-parameter logistic model assume?

A
  • It assumes that all scale items relate to the latent trait equally and items vary only in difficulty (equivalent to having equal factors loadings across items).
  • It also assumes that there is no chance of someone with a low latent ability guessing the correct answer.
34
Q

What happens if two ICCs have the same *b *parameter?

A

The two curves will intersect at 0.5.

35
Q

What three things does a computer need to be able to do in order for adaptive testing to work?

A
  1. Predict from the examinee’s previous responses how the examinee would respond to various test items not yet administered
  2. Make effective use of this knowledge to select the test item to be administered next
  3. Assign at the end of testing a numerical score that represents the ability of the examinee
36
Q

Please list some advantages of computerized adaptive testing.

A
  • Enhanced test security
  • Testing on demand
  • No need for answer sheets
  • Test pace that is keyed to the individual
  • Immediate test scoring and reporting
  • The minimization of test frustration for some examinees
  • Greater test standardization
  • Easy removal of “defective items” from the item bank when they are identified
  • Greater flexibility in the choice of item formats
  • Reduction in test supervision time
37
Q

What are the 6 areas in which research is being conducted in regards to adaptive testing?

A
  1. Choice of IRT model
  2. Item bank
  3. Starting point for testing
  4. Selection of subsequent test items
  5. Scoring/ability estimation
  6. Choice of method for deciding when to terminate the test administration
38
Q

What are the two procedures that are commonly used for item selection in adaptive testing?

A
  • Maximum information - selects items that provide maximum information (i.e., minimizes the standard error)
  • Bayesion item selection - selects test items that minimize the variance of the posterior distribution of the examinee’s ability
39
Q

Which IRT model is most appropriate in adaptive testing?

A

The three-parameter logistic model (3PL) because it includes a parameter for guessing (c)

40
Q

What are the two most commonly used estimation procedures when obtaining ability estimates in adaptive testing?

A
  • Maximum likelihood - problematic when the # of test items is small
  • Bayesion - may produce biased estimes of ability if inappropriate prior distributions are chosen
41
Q

When does computerized adaptive testing compute an initial ability estimate?

A

After the examinee has answered at least one item correct AND one item incorrect

E.g., Item 1 = 1, Item 2 = 0
E.g., Item 1 = 1, Item 2 = 1, Item 3 = 0
E.g., Item 1 = 0, Item 2 = 0, Item 3 = 0, Item 4 = 1

42
Q

When does computerized adaptive testing stop?

A

Once the standard error of the examinee’s ability estimate stops decreasing by a specified amount.

E.g., 0.05

43
Q

Are items with larger a parameters (steeper slopes) better for discrminating among examinees near the same ability level Θ than items with smaller a parameters (less steep slopes)?

A

Yes!

44
Q

a. What is the range of values that the *a *parameter (discrimination) can theoretically take on?
b. What is the range of values that the *a *parameter usually takes on?

A

a. (-∞, +∞)

∞ = infinity

However, negatively discriminating items are discarded from ability tests because something is wrong with an item if the probability of answering it correctly decreases as examinee ability increases (i.e., negative correlation between the latent trait and the probability of answering it correctly).

b. (0,2) - it is unusual to see values of a that exceed 2

45
Q

What assumption does the two-parameter logistic model make?

A

That there is no chance that someone guessing will get an item correct. This assumption typically is held when items are free response, but it can also hold for multiple choice items if the items are easy OR if the test follows good instruction.

46
Q

What is the lower asymptote for both the one- and two-parameter logistics models?

A

Zero (because there is no parameter for guessing [c])

47
Q

Describe how the 2PL model is simply a special case of the 3PL model?

A

The 2PL model is a 3PL in which the c parameter is set to zero.

48
Q

Describe how to 1PL model is simply a special case of the 3PL model?

A

It is a 3PL model in which the c parameter is set to zero and the a parameter is set to 1 (or any other constant) for all items.

49
Q

When is a 2PL model typically used?

A
  • When the there is little to no chance of a person low on the latent trait guessing the correct response for an item. This model is typically used for free response items, OR…
  • When there is no correct answer (i.e., the concept of guessing does not apply) such as in personality tests
50
Q

Which IRT model is used when there are too few examinees to get high quality estimates of the a and c parameters?

A

The 1PL model.

51
Q

Will ICCs created using a 1PL model ever cross?

A

Nope!

52
Q

When does Lord’s paradox occur?

A

It occurs when examinees who are lower in ability have a higher probability of answering an easier question correctly compared to those who are higher in ability, BUT they also have a lower probability of answering a harder question correctly than those who are higher in ability.

53
Q

What are two advantages of using a Rasch model over IRT?

A
  1. One is the primacy of Rasch’s specific requirements, which (when met) provides fundamental person-free measurement (where persons and items can be mapped onto the same invariant scale).
  2. Estimation of parameters is more straightforward in Rasch models due to the presence of sufficient statistics, which in this application means a one-to-one mapping of raw number-correct scores to Rasch Θ estimates
54
Q

What are two reasons ICCs curves are useful besides item comparison?

A
  • Comparing groups to assess item bias
  • Linking scores over time
55
Q

a. What is the theoretical range of the c parameter?
b. What is the typical range of c parameter values?

A

a. 0 to 1.0
b. < 0.3

56
Q

What do higher *c *parameters represent?

A

A higher minimum asymptote (lowest probability of selecting a response) for that item’s ICC.

57
Q

Which parameters are typically not present in non-cognitive latent traits?

A

Parameters c and d

58
Q

What is the formula for calculating the *b *parameter?

A

p(b) = (1+c)/2

That is, it is the halfway point between c<em>i</em>* *(lower asymptote) and 1 (higher asymptote)

59
Q

Why is the value of

A
60
Q

When the *c *parameter is 0, what is an easy way to find out the *b *parameter by looking at a ICC?

A

See what the latent trait/ability is when the curve passes through 0.5.

61
Q

What assumption must be met in order to determine the likelihood function for an examinee?

A

The assumption of local independence.

Psychometrics (6 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions