Item analysis Flashcards
What are the 2 basic ways items can differ?
They can differ on item difficulty and item discrimination
Item difficulty
magnitude of responsiveness
Item difficult for polytomous items
(more than 2 response options)
Means and SDs show an index of difficulty
Item difficulty for dichotomous items
(2 response options)
Difficulty is defined by the number of people who get a particular item correct or choose a particular option.
E.g. if 74% get it right, item difficulty (p value) is p=.74
:( extreme p values (close to 1 or 0) are limited because they don’t allow for variability if all get it right/ wrong (too easy or too hard)- there is poor discrimination between people
Item difficulty- variance for dichotomous items
p x q
p= number who got it right q= number who got it wrong
e.g. difficulty=.8 (80% got it right)
p=.8, so q=.2
.8x.2= .16
so variance = .16
Item discrimination
Compares the relationship between performance on an item an performance on the whole test
Extreme group method
(A way to measure item discrimination)
1) Take the top and bottom 1/3 of test scores
2) For each item- see who got it right
3) Subtract those ho go it right in the lower group from those who got it right in the top group
d= p (upper)- p (lower)
d value- discrimination index
d>0 shows some discrimination, negative values shows poor discrimination
Item-total correlation
(A way to measure item discrimination)
Correlation between the score on an item (i.e. right or wrong) and total test score
High positive correlation means the item discriminates well
Negative correlations or those around 0 are problematic and should be remove
For tests with low number of items, remove them before correlating with the rest so they are not correlating with themselves (e.g. correlate item 1 against items 2-5)
Item characteristic curve (ICC)
(A way to measure item discrimination )
Graphical representation
Y axis- proportion of people who got the item right
X axis- total test score (people put into groups of scores e.g. score 21-30, 31-40 etc.)
Could also be the probability or getting an item correct against the level of the construct being measured
:) gives an overviews of the difficulty and discrimination parameters
Interpreting an ICC
Difficulty of item- shown by it’s location on the X and Y axis
Discrimination- shown by the slope of ICC
Steeper slope= more discrimination
Items with higher scores are easier than those with lower scores
Parameter names in ICC
discrimination= a parameter
difficulty= b parameter
