Psychometrics Flashcards
What determines choice of format in item writing? (2 marks)
Objectives and purposes of test (eg do we want to measure extent/amount of interaction, or quality of interaction)
Difference between objective and purpose of a study?
Purpose - broad goal of research
Objective - how are we practically going to achieve that
List 4 of the 9 item writing guidelines
- clearly define what you want to measure
- generate an item pool (best items are selected after analysis)
- Avoid long items
- Keep the reading difficulty appropriate
- use clear and concise wording (avoid double-barrelled items and double negatives)
- Use both pos & neg worded items
- use culturally neutral items
- (for MCQS) - make all distractors plausible & vary position of correct answer
- (for true/false Qs) - equal numbers of both and make both statements the same lenth
List the 5 categories of item formats
- Dichotomous
- Polytomous
- The Likert format
- The Category format
- Checklists and Q-sorts
Advantage of the dichotomous format (3 marks)
- easy to administer
- quick to score
- requires absolute judgement
Disadvantages of the the dichotomous format (3 marks)
- less reliable (50% chance of correct answer)
- encourages memorization instead of understanding
- often the truth is not black and white (true false is an oversimplification)
Minimum number of options for a polytomous format?
3 (but 4 is commonly used, and considered more reliable)
3 guidelines in writing distractors in the polytomous format
- distractors must be clearly written
- distractors must be plausible correct answer
- avoid “cute” distractors
Advantages of polytomous questions (4 marks)
- easy to administer
- easy to score
- requires absolute judgement
- more reliable than dichotomous (less chance of guessing correctly)
Formula for correcting guessing
R-(W/(n-1))
Fields in which Likert scales are predominantly used (2 marks)
Attitude and Personality questionnaires
How can one avoid a the neutral response bias in Likert Scales
have an even number of options
How does one score negatively worded items from a Likert scale
Reverse score
Suggested best no. of options in a category format question?
7
Disadvantages of the category format (2 marks)
- tendency to spread answers across all categories
- susceptible to the groupings of things being rated (rate an item lower if those other items in the category are really good - i.e not objective)
When best to use category format questions? (2 marks)
- when people are highly involved in a subject (more motivated to make a finer discrimination)
- when you want to measure the amount of something (eg levels of road rage)
Two tips when using the category format
- make sure your endpoints are clearly defined
- use a visual analogue scale (ideal with kids, for e.g smily face on one side of scale and frowny face on the other to describe how they’re feeling)
Where are Checklists format questions commonly found?
Personality measures (e.g a list of adjectives, tick those that describe you)
Describe the process of Q-sort format questions
Place statements into piles, piles indicate the degree to which you think a statement describes a person/yourself
In terms of Item analysis, describe item difficulty and give another name for it
The proportion of people who get a particular item correct (higher value = easier item)
AKA facility index
p = no of correct answers/no of participants
Ideal range for optimum difficulty level
0,3 - 0,7
How to calculate ODL (optimum difficulty level) for an item
Halfway between 100% the chance of guessing the answer correctly (1+chance)/2
E.g: For a item with 4 options, ODL = (1+0,25)/2 = 0.625
How should difficulty levels range across items in a questionnaire
You want most items around the ODL and a few at the extremes. The distribution of p-values (difficulty levels) should be approximately normal
Why does one need a range of item difficulty levels?
To discriminate between ability of test-takers