Surveys Flashcards
A survey is…
a system for collecting information from or about people to describe, compare, or explain their knowledge, attitudes, and behavior (Fink, 2003).
can be quantitative & qualitative
Key characteristics of a survey
- Strives for a complete picture, using a (representative) sample of a population.
- Relatively large N; standardized data from a large number of objects
- Often, surveys measure independent and dependent variables in one go (cross-sectional)
Use of a survey
- Testing specific hypotheses?
- Abstract or tangible concepts that are not readily available?
- Can the answers to your questions be quantified (ratings/scores)?
- Can entities at your level of analysis complete surveys?
- Do you need data from a lot of potential respondents?
5x yes? -> Surveys are a good tool to collect data
What is your population?
What kind of entities do you want to say something about?
• Which characteristics do they have?
• Which characteristics do they not have?
• How broad/narrow to focus?
• Generalizability vs. Specificity
How to sample?
- What kind
- How to target
- How many to target
- What to offer in exchange
What kind of respondents can help you answer the RQ?
- Representativeness
* Selection biases
How to target respondents with those characteristics?
- Convenience vs. Random sampling
* Stratified sampling (comparing groups)
How many to target?
- Response rates
- Non-response biases
- Response quality
How many surveys do you need?
General rules of thumb
- 20 responses per variable (construct) collected by survey
* 30-50 per experimental condition
How many surveys do you need?
Take into account…
- Response Rate
• FB/Twitter have very low response rates
• Personalized e-mails have higher response rates than general e-mails.
• Make sure to exert effort to increase the response rate (and decrease non-response bias) - Data Quality
The more ‘committed’ an individual is to the project, the higher the data quality.
Abstract constructs
- Use previously published/validated scales from literature
- Do not come up with your own questions (if needed adapt from other sources)
- Multiple questions for each construct (typically 3 or more)
- Use of Likert scales
How many response options?
- Typically between 5 & 7 points
- Use original scale from the source
- Be consistent
Tangible constructs
- use single question
- do not ask when you can collect from other sources
- do not frame questions
- do not ask questions you don’t need
Open-ended questions
- E.g. asking respondents to state five challenging things in their job.
- Only if new/unknown input is needed
Closed questions
- E.g. list 10 or 15 aspects that might seem challenging in jobs and ask the respondents to rank the first five among these.
- Closed questions help respondents make quick decisions.
- They help the researcher to code the information easily for analysis.
A double-barreled question
A question that lends itself to different possible responses to its subparts
Such questions should be avoided and two or more separate questions asked instead.
Non-neutral questions
Disagreeing can mean two things
- not very satisfied (but still satisfied)
- dissatisfied
Avoid loaded wording (e.g. awful, amazing)
Social desirability tendency
Sometimes certain items that tap social desirability are deliberately introduced at various points in the questionnaire and an index of each individual’s social desirability tendency is calculated therefrom. This index is then applied to all other responses given by the individual in order to adjust for social desirability bias.
