Ch 8.1 Test Development

Question 1

Test Conceptualization

Answer 1

Coming up with an idea for a test

Why is a test needed?

• Existing measures are not good enough
• There are no existing measures (e.g. new medical disease)
• Literature review

Question 2

Norm - referenced

Answer 2

• Distinguish between high & low scorers
• Relative standing
• EX: WAIS-IV

Question 3

Criterion - referenced

Answer 3

• Distinguish between mastery and non - mastery
• Pass/fail
• Ex: Driver’s license exam

Question 4

Writing Items

Answer 4

Need to develop item pool

• Reservoir from which items will be drawn or discarded for the final version of the test
• Twice as many as the final version

Question 5

Where do items come from?

Answer 5

• Literature reviews
• Expert opinion
• Personal experience, knowledge

Question 6

Question Interpretation

Answer 6

Matching what you think you are asking wo what participants think you are asking

Question 7

Considerations for writing questions

Answer 7

-Be careful about eliciting a certain type of response

• Avoid leading questions
• Double barreled questions

Question 8

Rules for writing questions

Answer 8

• Follow the rules of communication
• Provide sufficient instructions
• Allow for variability

– Make measurement scale meaningful

– Floor and ceiling effects

Question 9

Survey Formats [placeholder]

Answer 9

.

Question 10

Forced Choice

Answer 10

Questions require respondents to select from a limited number of answers

Question 11

Categorical Responses

Answer 11

• Dichotomous questions
• Nominal response format
• Ordinal responses using rank order

Question 12

Interval level response

Answer 12

• Likert scale
• Semantic differential
• Response brackets
• Guttman scale or cumulative

Question 13

Guttman scale or Cumulative

Answer 13

• Items range sequentially from weaker to stronger expressions of the construct being measured

Question 14

Open-ended (free or constructed response)

Answer 14

• Fill in the blank
• Short answer

Question 15

Filter or contingency questions

Answer 15

• Do not ask questions that cannot be answered by all participants

Question 16

Scoring and Scaling

Answer 16

Scoring

• One vs multiple raters
• Partial credit vs correct/incorrect
• Are instructions or training required?

Unit Weighting

Or Weighting by:

• item difficulty
• effort or time spent
• reliability (importance)

Question 17

Test Tryout [placeholder]

Answer 17

.

Question 18

Piloting

Answer 18

Pilot - (work, study, research)

• Preliminary test surrounding the creation of a prototype of the test
• Should items be included in the final version?
• Determine the best measurement methodology

Question 19

Test Tryout

Answer 19

• Tryout with people similar to those for whom test was designed
• 5 - 10 people for each item
• Should items be kept, revised, or discarded?
• Use standardized conditions