Chapter 6 - Writing Test Items

Question 1

DEVELLIS

Answer 1

Provided guidelines for writing
Define what you want to measure
Generate item pool
Avoid long items
Avoid double barrelled item with two ideas
Mix positive and negative worded Items
-acquiescence response set

Question 2

Acquiescence response set

Answer 2

People tend to agree with test items

-SO word them on opposite directions

Question 3

Dichotomous Format

Answer 3

• true/false
• can easily construct from text
• simple, quick, easy to score
• but encourages memorization
• truth comes in grey
• personality tests often use

Question 4

Distractors

Answer 4

• the wrong answers in polytomous (multiple choice) test
• rare that more than 4 are efficient
• ineffective/joke ones hurt reliability -time consuming, losers guess right
• best to have 3-4 good ones

Question 5

Likert Format

Answer 5

• test that requires person to indicate degree of agreement
• scoring needed negatively worded items to be reverse scored, then responses summed
• popular in measuring attitude
• can be subjected to factor analysis so developers should find groups of items that go together
• Format used to create likert scales

Question 6

Category Format

Answer 6

• 10 point rating scale
• controversy: people change ratings depending on context AND subjects spread event across 10 points
• more reliable if all options clearly rated rather than the extremes labelled

Question 7

Visual Analogue Scale

Answer 7

• given a line and asked to mark place between two defined end points
• popular for measuring self health
• not used for multi item scales because scoring is time consuming

Question 8

Checklist

Answer 8

• test form common in personality measurements with adjective checklist
• must endorse an adjective or not

Question 9

Q Sort

Answer 9

• test form used to described self or provide ratings of others
• given statement and asked to sort into 9 piles
• hit home pile 9, not at all pile 1
• most put around 5 making bell curve

Question 10

Item Difficulty

Answer 10

• number of people who get item correct
  1. Determine probability item could be answered by chance (.50 T/F, .25 for MC)
  2. Add 1.00 to chance performance and divide by 2.0

Question 11

Optimal Item Difficulty Level

Answer 11

Halfway between 100% of people and level of success expected by chance alone
-ex. ODL for 4 multiple choice is .625

Question 12

Extreme Group Method

Answer 12

Compared people who have done well with those who suck

-difference between these is discrimination index

Question 13

Discrimination Index

Answer 13

Difference between people proportion of people who got answer right with those that didn’t

Question 14

Point Biserial Method

Question 15

Point Biserial correlation

Answer 15

Correlation beteeen dichotomous variable and continuous variable

Question 16

Item Characteristic Curve

Answer 16

Can prepare graph for each test item

• total test score is plotted on horizontal x
• proportion of examiners who get the item correct plotted on vertical y

Question 17

Item Response Theory

Answer 17

Newer approaches to test based on item analysis consider chances of getting particular items right or wrong

• each item on test has its own item characteristic curve that describes probability of getting each answer right given ability level
• some say it’s most important development in testing

Question 18

PROMIS

Answer 18

Patient Reported Outcome Measurment Information Systems

• to develop precise measures that described how patients report their health status
• designed to provide practicing docs with highly valid measures that can be used in clinical practice research
• large item bank created

Question 19

PROMIS

Answer 19

Patient Reported Outcome Measurment Information Systems

• to develop precise measures that described how patients report their health status
• designed to provide practicing docs with highly valid measures that can be used in clinical practice research
• large item bank created

Question 20

Criterion Reference Test

Answer 20

Compared performance with some clearly defined criterion for learning

• popular in individualized instruction programs
• many regard CRT as diagnostic instruments to
• if student does poorly on some items, teacher knows individual education needs more focus in one area

Question 21

Antimode

Answer 21

Least frequent score on frequency polygon

• made when evaluating test items
• item is given to two groups of students (one that’s been taught material, one that hasnt)
• V polygon results: scores on right from inexperienced, scored on left from experience led
• BOTTOM OF V IS ANTIMODE: least frequent score!
• this point divides those who’ve been exposed from those who haven’t
• taken as CUTTING SCORE
• when people score higher, they’ve met objective