Midterm Flashcards
What are the main decisions when designing research instruments?
- Appropriate type of questions
- Make sure wording of questions, instructions and cover letter is clear
- Decide method of distribution and return of the questionnaires
- Determine methods of data collection
What are the three types of question content?
Behaviour
Beliefs and attitudes
Characteristics and attributes
What are the types of questions?
Direct
Indirect
Open-ended
Closed
When do you use direct vs indirect questions?
Use indirect when there is a sensitive topic
Typically use direct
What are the types of open-ended questions?
Infinite responses
Limited amount of answers
Starting questions
Follow ups
Questions bringing arguments
Questions bringing attitudes
What are the ways that answers can be recorded by interviewee?
Lists
Categories
Rankings
Scales
Grids
Quantities
Describe a list question
Multiple choices that are not mutually exclusive
“Check all that apply”
Describe a category question
Multiple options but all are mutually exclusive (e.g. at least once a day, at least once a week…)
“Choose only one option”
Describe a ranking question
Rank between 1 and 5 for example, placing numbers beside choices as per the instructions
Describe a scale question
Usually horizontally formatted
From strongly agree to strongly disagree for example
Describe a grid question
Multiple scale questions in one grid
Describe a quantity question
Question with numerical answer such as amount of times you have done something, or the year you were born etc.
What are the key response bias issues in survey research?
Non-differentiation
Central tendency
Leniency
Auspices
Telescoping
Fatigue
What is non-differentiation?
When a respondent selects the same answer for all questions
What is the central tendency issue?
It is the tendency of respondents to select answers in the middle of a scale question
How should you solve a central tendency issue?
You should expand the scale
(It is shown that reliability and validity highest for about a 7 point scale for bipolar scales, and 5 points for unipolar)
What is the leniency issue?
The tendency to give inflated ratings rather than true assessments (common in performance ratings of employees etc.)
How can you solve the leniency issue?
By shifting the scale down, for example:
instead of
not helpful at all - of some help - very helpful
use
of some help - very helpful - absolutely indispensable
What is auspices bias?
Bias in the responses of subjects caused by the respondents being influenced by the organization conducting the study
What is the telescoping issue?
A bias causing people to misremember the time frame of events
How can you solve telescoping bias?
By expanding the time frame in question
What is the fatigue issue?
When a respondents answers are affected due to the fact that the survey is too long/complex
How do you solve the fatigue issue?
Keep the survey short and to the point
What is the data on the fatigue issue in surveys?
It is shown that surveys that took more than 17.5 minutes led to predicted completion rates of less than 70%
Surveys with more than 30 screens/questions are predicted to exceed acceptable dropout levels
Other reports indicate that 55 clicks is a threshold level
What is the social desirability response bias?
- A response determinant that refers to the tendency of people to deny socially undesirable traits or qualities and to admit to socially desirable ones
- Leads to over-reporting of socially acceptable attitude/behaviours and under-reporting of the undesirable one
How does social desirability differ or remain the same across cultures?
The concept of social desirability is the same but the content of it varies
Name some things that may be over-reported due to the social desirability bias.
Good citizenship
Being cultured
Helping others
Abiding by the law
Name some things that may be under-reported due to social desirability bias.
Illness
Consumption of alcohol
Illegal behaviours
Financial status
Can declared attitudes be interpreted as a predictor of real attitudes?
No
How can we reduce social desirability bias?
- Change the overall context i.e. explain that there are not good or bad answers etc.
- Utilization of Marlowe-Crowne social desirability scales
- Change the research design i.e. eliminate the interviewer and use SAQ etc.
- Triangulation i.e. use of different data sources on respondents and their behaviours
What are the potential problems with utilizing social desirability scales in reducing SDRB?
It does’t inform about the impact of SDRB on particular answer
What are the potential problems with changing the research design to reduce SDRB?
- Its not always applicable
- Can make organization of field works difficult
- It is hard to prove the effect of such provisions
What are the potential problems with triangulation as a way to reduce SDRB?
Alternate data may not always be available
There may be time pressure and budgetary constraints
The credibility of the data sources may be varying or even unknown
What is acquiescence bias?
A category of response bias that results because some individuals tend to agree with all questions or to concur with a particular position
When is acquiescence most likely to happen?
- When a question is difficult to answer
- After respondents have become fatigued
- During telephone interviews rather than face-to-face
What are possible explanations for acquiescence bias?
- Desire to be polite
- Desire of lower social status people to defer to higher social status individuals
- Inclination to satisfice rather than optimize
What are the two extremity biases? Describe them
The primacy effect - people’s tendency to better remember the first piece of information they receive (common in self administered questionnaires)
The recency effect - people’s tendency to better remember the last/most recent piece of information they receive (common when answer options being read out by interviewer)
What are the cognitive steps in answering questions
- Understand question
- Search memory for information
- Integrate information into summary judgement
- Translate judgement onto response alternatives
What are the models of decision making?
Rational model
Normative model
Describe the rational model
- Problem is clear and unambiguous
- Single and well-defined goal can be identified
- Alternatives and consequences are all known
- Preferences and needs are clear, recognized, and unchanging
- Time and resources are abundant and accessible
Describe the normative model
- Based on premise that decision making isn’t rational
- Decision makers are guided by bounded rationality
- Decision-making is, among others, characterized by satisficing
What is satisficing?
- Satisfying and sufficing
- Respondents not carrying out survey with the level of attention researchers expect
- Explanation of how acquiescence is caused
- “When faced with a demanding information-processing task, people often expend only the effort necessary to make a satisfactory or acceptable decision”
What happens in weak satisficing?
- Skip understand question and translate judgement (steps 1 & 4) and only do search memory and integrate info (steps 2 & 3)
- Don’t properly read the question and so may not know how to answer follow-up, leading them to choose what they think you want to hear
What is strong satisficing?
- Only understanding question and translating judgement (steps 1 & 4) and skipping search memory and integrate info (steps 2 & 3)
- Response is just a mere estimation as they don’t try to retrieve info from memory
- Response may be what they think the researcher wants to hear (not aligning with the actual memories of the interviewee)
What is complete satisficing?
Only translating judgement (step 4) and skipping steps 1-3
How do you detect complete satisficing?
The following properties of questions are important to note:
- devoid of the need of info retrieval and integration (don’t need steps 2 & 3)
- selection of the correct answer solely depends on question comprehension (step 1 = step 4)
- no domain-specific knowledge or interests in question interpretation
- low cognitive demand, similar to instructions
What are the regulators of satisficing/what can cause it to happen?
- Difficulty of the task
- Respondents ability to perform the required task
- Respondents motivation to perform the task
What are the two types of usual answers to open-ended questions?
- No answers
- Unpublishable
