Writing and Evaluating Test Items Flashcards
What’s the most common used method in psychological testing?
Questionnaires
Questionnaire characteristics (3)
(1) Written series of questions
(2) Structured stimuli (i.e. questions)
(3) Structured responses (i.e. response format)
Questionnaires Advantages (4)
(1) Presentation of stimuli is well controlled
(2) Scoring highly reliable
(3) Efficient to administer to large numbers
(4) Inexpensive
Example of Questionnaires with Dichotomous Choice Formats
Checklist; True-False and Agree-Disagree
Rating Scales Formats def
Indicate the extent of occurence
E.g. During the last month, how often have you experienced the following?
What’s the problem with rating scales formats?
Number of points? How many options?
More options = more variability
At what point is it too much?
Middle point? Acknowledges that pple might NOT have an opinion, but can be an easy way out
How many points are optimal for a likert scale?
If likert scale, 7 = optimal
Forced Choice Formats
Person is presented with 2 to 4 stimuli and asked to choose among them.
Q-sort Formats
Forced distribution of items into categories
E.g. E.g. Give person list of 100 characteristics. Group the characteristics according to how like the person the characteristics are:
Four Steps in the Question-Answer Process
(1) Comprehension: Attending to questions and instructions
(2) Retrieval: Retrieval of relevant information
(3) Judgment: Integration of retrieved information
(4) Response: Mapping the judgment on the response category
Issues for Questionnaires (3)
Response set: Tendency for people to respond to questions in a way that paints a certain picture of themselves instead of providing honest answers
(1) Acquiescence = Tendency to agree, say true, say often.
(2) Social desirability = Tendency to present self in a socially favorable manner
(3) Random responding = Ignoring or paying insufficient attention to item content
Errors in completing a questionnaire = ____ (def)
Formally valid answers that do NOT reflect true scores, undermining data quality
How can we combat Acquiescence bias?
Use reverse-score items
What are the 2 components of Social desirability?
Impression management & Self deception
How can we combat social desirability? (3)
(1) Measure influence: assess discriminant validity
(2) Marlowe-Crowne social desirability scale
(3) Change response format (forced choice; Q-sort)