Midterm 2 - Writing/Evaluating Test Items (Ch. 6)

Question 1

What is the most commonly used method in psychological testing?

Answer 1

questionnaires!

Question 2

(T/F) questionnaires are always self-report

Answer 2

FALSE

Question 3

What 2 examples were given of questionnaires that are not self-report?

Answer 3

• Hamilton Depression Scale-21 (clinician ratings)
• Child Behavior Checklist (used by parents)

Question 4

In a questionnaire, the stimuli are (un/structured) and the responses are (un/structured)

Answer 4

both are structured

Question 5

What are 4 key advantages of the questionnaire method generally?

Answer 5

• presentation of stimuli well controlled
• scoring is highly reliable
• efficient to admin to large numbers
• inexpensive

Question 6

What are the 2 main types of dichotomous choice formats?

Answer 6

• checklist (no report of degree)
• true/false or agree/disagree

Question 7

What do we call a Likert-type scale?

Answer 7

rating scale (type of questionnaire)

Question 8

Rating scales are considered an (interval/ordinal) scale

Answer 8

big debate, no consensus

Question 9

How many categories should ideally be presented in a rating scale type questionnaire?

Answer 9

5-7 (hard to have clear and reliable labels beyond 7)

Question 10

Should there be a middle point on rating scales?

Answer 10

depends on type of question!

Question 11

What is a forced-choice format questionnaire? What was the main subtype presented in class?

Answer 11

• presented w 2-4 stimuli and asked to choose (eg MCQ)
• no middle! (have to pick one)
• ex Q-sort format

Question 12

What is a Q-sort format questionnaire? What is it often used for?

Answer 12

• forced distribution of items into categories
• often used for personality tests
• ex have to rate each symptom in order of most to least frequent (none can be equal)
• ex forced normal distribution for list of 100 characteristics
• helps make fine distinctions bc you can’t just “agree” with everything or rate all things high

Question 13

What are the 4 main issues for questionnaires?

Answer 13

• response set (eg acquiescence)
• social desirability
• faking
• random responding

Question 14

What is the response set issue for questionnaires? How can we combat this?

Answer 14

• tendency to respond in a certain way
• creates systematic error
• eg acquiescence (tendency to agree w everything)
• combat using reverse-score items

Question 15

What is the social desirability issue for questionnaires (2 components)? How can we combat this?

Answer 15

• impression management and self-deception
• can use Marlowe-Crowne measure of social desirability and assess discriminant validity
• can use forced-choice/Q-sort format

Question 16

What is the faking issue for questionnaires?

Answer 16

• answering in a way that will result in a specific outcome
• can be pos or neg

Question 17

What is the random responding issue for questionnaires? How can we combat this?

Answer 17

• responding randomly
• can use distractors to identify this

Question 18

Fill in the blanks for these 12 tips for writing good items:

• ___ idea(s) per item stem
• write each item in a ___ and ___ manner
• avoid ___ items
• avoid _____
• ____ appropriate for intended test-takers
• avoid ___ or ___ language
• make all items ___
• ask someone else to ____
• make all responses similar in ___ and ___
• make sure the item has only one ____
• avoid words such as ___ and ___
• avoid ____ responses

Answer 18

• ONE idea(s) per item stem
• write each item in a CLEAR and DIRECT manner
• avoid LONG items
• avoid DOUBLE NEGATIVES
• READING LEVEL appropriate for intended test-takers
• avoid SLANG or COLLOQUIAL language
• make all items INDEPENDENT
• ask someone else to REVIEW
• make all responses similar in LENGTH and DETAL
• make sure the item has only one BEST ANSWER
• avoid words such as ALWAYS and NEVER
• avoid OVERLAPPING responses