Writing and Evaluating Test Items Flashcards

1
Q

What’s the most common used method in psychological testing?

A

Questionnaires

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Questionnaire characteristics (3)

A

(1) Written series of questions
(2) Structured stimuli (i.e. questions)
(3) Structured responses (i.e. response format)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Questionnaires Advantages (4)

A

(1) Presentation of stimuli is well controlled
(2) Scoring highly reliable
(3) Efficient to administer to large numbers
(4) Inexpensive

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Example of Questionnaires with Dichotomous Choice Formats

A

Checklist; True-False and Agree-Disagree

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Rating Scales Formats def

A

Indicate the extent of occurence
E.g. During the last month, how often have you experienced the following?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What’s the problem with rating scales formats?

A

Number of points? How many options?
More options = more variability
At what point is it too much?
Middle point? Acknowledges that pple might NOT have an opinion, but can be an easy way out

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

How many points are optimal for a likert scale?

A

If likert scale, 7 = optimal

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Forced Choice Formats

A

Person is presented with 2 to 4 stimuli and asked to choose among them.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Q-sort Formats

A

Forced distribution of items into categories
E.g. E.g. Give person list of 100 characteristics. Group the characteristics according to how like the person the characteristics are:

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Four Steps in the Question-Answer Process

A

(1) Comprehension: Attending to questions and instructions
(2) Retrieval: Retrieval of relevant information
(3) Judgment: Integration of retrieved information
(4) Response: Mapping the judgment on the response category

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Issues for Questionnaires (3)

A

Response set: Tendency for people to respond to questions in a way that paints a certain picture of themselves instead of providing honest answers
(1) Acquiescence = Tendency to agree, say true, say often.
(2) Social desirability = Tendency to present self in a socially favorable manner
(3) Random responding = Ignoring or paying insufficient attention to item content

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Errors in completing a questionnaire = ____ (def)

A

Formally valid answers that do NOT reflect true scores, undermining data quality

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

How can we combat Acquiescence bias?

A

Use reverse-score items

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What are the 2 components of Social desirability?

A

Impression management & Self deception

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

How can we combat social desirability? (3)

A

(1) Measure influence: assess discriminant validity
(2) Marlowe-Crowne social desirability scale
(3) Change response format (forced choice; Q-sort)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Random responding occurs when respondents ____ (3)

A

(1) Do not read an item
(2) Do not understand an item
(3) Are unmotivated to think about the item

17
Q

How can we detect random responding? (4)

A

(1) Instructed response items: Ask for a specific answer (e.g. “choose strongly disagree”)
(2) Bogus items: Ask about impossible or improbable scenarios (e.g. “I was born before 1920”)
(3) Self-report items: Ask participants about their care and engagement DURING the survey
(4) Response time: Computed after data collection but must be considered before starting

18
Q

Writing Good items

A
  • Single idea per item stem
  • Write each item in a clear and direct manner
  • Avoid long items
  • Avoid double negatives
  • Reading level appropriate for intended test-takers
  • Avoid slang or colloquial language
  • Make all items independent
  • Ask someone else to review items to reduce ambiguity and inaccuracies
  • Make all responses similar in length and detail
  • Make sure the item has only one best answer
  • Avoid words such as “always” and “never”
  • Avoid overlapping responses