Surveys 6.2 & 6.3 Flashcards
In-person advantages
- ppts can ask for clarification
* relatively high response rate
In-person disadvantages
- relatively more expensive
- takes longer to collect the data
- potential for interviewer bias
- Use trained interviewers
- increased risk of socially desirable responses
- Convince ppts data will be anonymous and use trained interviewers
- Sampling bias – respondents not representative of target population
Self-administered surveys advantages
- Relatively inexpensive
* Easy to distribute
Self-administered disadvantages
- Lower response rate
- Sampling bias
- Make the survey as accessible as possible (e.g., email, social media…)
- Ppts may not be who they claim to be
- Ppts can’t ask for clarification
- May still get socially desirable responses but less likely than in-person
- No interviewer bias but still need to worry about biased questions
Practices for reducing bias
•Use trained interviewers •Make survey anonymous •Make the survey easy to access, understand and complete •Easily available to those representative of your population •Not easy to access for those who are NOT members of your target population •Overall format and structure •Questions are clearly written and unbiased
Satisficing
doing the minimum to complete a task
•For surveys, the minimum is “putting an answer”
Speed runs
Rushing through without paying attention
Straight lining
Giving identical answers on each question
•E.g., always selecting “strongly agree”
Addressing satisficing
•Think carefully about the incentives that are offered •Don’t make the survey too long •Include catch trials •Select A on this question •Ask similar questions •Check for consistency (internal reliability) •Switch the polarity of the questions •Strongly agree to Strongly disagree (I like cats I dislike kittens) •Switch the polarity of the anchors •Be careful, people may not notice or may get confused
Response burden
The effort required for a participant to complete a survey •The greater the burden, the less likely people will complete it •The less likely they will return for follow-ups
Layout of Survey Questions
The survey should appear relatively
easy and interesting
Topics should be organized so that they
are easy to follow and motivate ppts to
keep going
Structural organization
Goal: get participants to complete the survey and give you good data
Funnel structure:
Beginning
•broad, general interest questions that are easy to answer
•warm the ppt up and get them involved in the survey
Middle
•Questions with a narrower focus; and those that are more difficult, sensitive, or time
consuming
•Ppts have invested some time and they aren’t burnt out
End
•Demographic questions
•Easy to answer, important, but not all that interesting
•Also answering these questions can induce stereotype threat which could bias later responses
Branching using filter questions
When how you
respond on one question
influences the questions you will
see next. Useful to avoid redundant or irrelevant questions.
Filter questions: questions that lead to branching
Battery
•A set of questions that share an introduction and that all have the same response categories •Conceptual alignment makes it easier to think through and take
Open-ended questions
No structure provided for the answer
Benefits: •Ppts can include lots of details •Ppts can include explanations •Ppts may include info you had not thought to ask about
Drawbacks
•Coding the data
•Time consuming, difficult
