Rating Scales and Other Considerations for Surveys Flashcards
What is the purpose of rating scales in a survey?
To ask people to provide “how much” judgements
Rating scales are particularly useful for measuring…?
Attitudes
What are the different formats of rating scales? Give 6 examples
1) Dichotomous, multichotomous
2) Likert scale
3) Graphic rating scale
4) Non-verbal scale
5) Ranking scale
6) Semantic differential scale
What are dichotomous rating scales?
A two-point scale that presents options that are absolutely opposite to each other
A two-point scale that presents options that are absolutely opposite to each other
This is known as…?
Dichotomous rating scales
This type of response option does not give the respondent an opportunity to be neutral on his answer to a question
What type of rating scale is this?
Dichotomous rating scales
What are multichotomous rating scales?
Presenting a variety of options Ps could respond with and giving them the option to choose as either only one response or as many responses that apply to them
Presenting a variety of options Ps could respond with and giving them the option to choose as many that apply to them
This is known as…?
Multichotomous rating scales
What are Likert scales?
A rating scale used to measure opinions, attitudes, or behaviour, ranging from strongly disagree/agree, somewhat disagree/agree and neither disagree/agree
A rating scale used to measure opinions, attitudes, or behaviour, ranging from strongly disagree/agree, somewhat disagree/agree and neither disagree/agree
This is known as…?
Likert scale
What are graphic rating scales?
Respondents can choose a particular option on a line or scale to show how they feel about something, ranging from a scale of 1-3, 1-5 or 1-10
Respondents can choose a particular option on a line or scale to show how they feel about something, ranging from a scale of 1-3, 1-5 or 1-10
This is known as…?
Graphic rating scale
What are ranking scales?
Scales which force respondents to rank a list of items with only one selection in each rank
Scales which force respondents to rank a list of items with only one selection in each rank
This is known as…?
Ranking scales
What are semantic differential scales?
A scale that uses two bipolar adjectives, one negative adjective on the left of the scale, and one positive adjective on the right of the scale.
Participants indicate where they feel a product or experience lies on a 7-point scale
A scale that uses two bipolar adjectives, one negative adjective on the left of the scale, and one positive adjective on the right of the scale.
Participants indicate where they feel a product or experience lies on a 7-point scale
This is known as…?
Semantic differential scales
What type of rating scale is this?
Should there be a referendum for Welsh
independence?
Yes No
Dichotomous
What type of rating scale is this?
- Two response options
- The simplest type of quantification
Dichotomous
What type of rating scale is this?
How many hours did you spend revising last week?
- Less than 5 hours
- 5 to 10 hours
- 10 to 20 hours
- More than 20 hours
Multichotomous
What type of rating scale is this?
Which study activities did you engage in?
- watching podcasts
- making notes
- reading textbooks
- reading papers
Multichotomous
What type of rating scale is this?
It gives multiple options for only one response or gives multiple options for multiple responses
Multichotomous
What type of rating scale is this?
How attractive is your partner?
- Not at all attractive
- Somewhat attractive
- Reasonably attractive
- Very attractive
Likert scales
What type of rating scale is this?
- Consists of multi-point responses
- Aim to ensure equal spacing of response options
Likert scales
What should we keep in mind when using Likert scales? List 4 things
- Response acquiescence
- Potential introduction of double negatives
- Verbal responses for all, or only for anchors?
- Inclusion of neutral response?
What type of rating scale is this?
A scale that uses (happy - neutral - sad) emoticons and non-verbal response options
Non-verbal rating scale
What is a non-verbal rating scale?
A scale that uses (happy - neutral - sad) emoticons and non-verbal response options
What type of scale involves pointing to the face that shows your answer?
Non-verbal rating scale
What type of rating scale is this?
Measures the relative importance of several items
Ranking scale
What type of rating scale is this?
Rank the anxiety you experience in the following group situations (from 1 most anxious, to 4 least anxious)
Ranking scale
What type of rating scale is this?
Ps asked to mark along a continuous line which is anchored at each end
Graphic rating scale
What type of rating scale is this?
How would you rate the music lessons you received in school?
Not very enjoyable ————— Very enjoyable
Graphic rating scale
What type of rating scale is this?
Ps marked along a line and a score is recorded by measuring where the line was marked
Graphic rating scale
What type of rating scale is this?
More indirect measurement of attitude
Semantic differential scales
What type of rating scale is this?
Respondents indicate thoughts and feelings by marking a response on scales between bipolar opposite adjectives
Semantic differential scales
What type of rating scale is this?
Music lessons in school were:
Valuable : _ : _ : _ : _ : _ : Wasteful
Irrelevant : _ : _ : _ : _ : _ : Worthwhile
Semantic differential scales
What are survey response rates?
The percentage of questionnaires completed and returned from those distributed
The percentage of questionnaires completed and returned from those distributed
This is known as…?
Survey response rates
What are the 5 ways we can maximise survey response rates?
1) Keeping questionnaires short
2) Keeping questionnaires simple and clear
3) Including a pre-paid envelope with postal surveys
4) Sending a reminder (after one or two weeks)
5) Offering an incentive
What 4 things make a good questionnaire construction?
1) Questionnaires can measure one or more variables
2) Typically multiple items to a measure single variable
3) Variable score is calculated
4) Better measurements with more items
Why should we include multiple questionnaire items to measure a single variable?
To minimise the impact of response errors
What happens when there are too few response options in a questionnaire?
Results in low sensitivity
What happens when there are too many response options in a questionnaire?
Results in low reliability
What are psychometrics?
The science of measuring psychological constructs
The science of measuring psychological constructs
This is known as…?
Psychometrics
What 5 things do psychometrics measure?
1) Personality
2) Cognitive ability
3) Attitudes
4) Knowledge
5) Educational attainment
How is the quality of questionnaires assessed? List 2 criteria
1) Temporal consistency
2) Internal consistency
What is temporal consistency?
The extent to which our tool would provide the same results under the same conditions
The extent to which our tool would provide the same results under the same conditions
This is known as…?
Temporal consistency
What is internal consistency?
The extent to which our tool is measuring the construct we are interested in
The extent to which our tool is measuring the construct we are interested in
This is known as…?
Internal consistency
What is construct validity?
Is the construct we are trying to measure valid?
Does the construct itself exist?
How is the validity of a construct supported?
Supported by cumulative research evidence collected over time
Construct validity be assessed in terms of…? List 2 ways
1) Convergence validity
2) Discriminant validity
What is convergence validity?
Assesses whether the construct we are measuring correlates with tests of related constructs
What is discriminant validity?
Assess whether the construct we are measuring does not correlate with tests of different constructs
Assess whether the construct we are measuring does not correlate with tests of different constructs
This is known as…?
Discriminant validity
Assesses whether the construct we are measuring correlates with tests of related constructs
This is known as…?
Convergent validity
What are psychometric tests?
Standardised questionnaires/tests designed to measure particular traits or abilities
Standardised questionnaires/tests designed to measure particular traits or abilities
This is known as…?
Psychometric test
Give 3 examples of what psychometric tests might measure.
1) Personality inventories
2) Cognitive ability tests
3) Measure of mental health status
True or False?
Psychometric tests have norms available allowing for the interpretation of individual
participants data
True
It should not be compulsory to attend all lectures
1) strongly agree 2) agree 3) neither agree nor disagree 4) disagree 5) strongly disagree
a. What is the problem with this question?
b. What is the response format?
a. Double-negative
b. Likert scale
Could you be attracted to someone who earns less or has a lower-status job than you?
Yes/No
a. What is the problem with this question?
b. What is the response format being used?
a. Double-barrelled
b. Dichotomous
The relationship between my effort and my grades at university is otrhogonal (mark your response on the line)
Strongly agree____________________Strongly disagree
a. What is the problem with this question?
b. What is the response format being used?
a. Jargon
b. Graphic rating scale
What damage do you think is done to children who play violent computer games?
__________________________________________________________________
__________________________________________________________________
a. What is the problem with this question?
b. What is the response format?
a. Leading
b. Open response
Listening to music makes me feel (mark all that apply)
Energised; Inspired; Relaxed; Focussed
a. What is the problem with this question?
b. What is the response format being used?
a. Should be an open question
b. Multichotomous, multiple response
