Surveys

Question 1

A survey is…

Answer 1

a system for collecting information from or about people to describe, compare, or explain their knowledge, attitudes, and behavior (Fink, 2003).

can be quantitative & qualitative

Question 2

Key characteristics of a survey

Answer 2

• Strives for a complete picture, using a (representative) sample of a population.
• Relatively large N; standardized data from a large number of objects
• Often, surveys measure independent and dependent variables in one go (cross-sectional)

Question 3

Use of a survey

Answer 3

1. Testing specific hypotheses?
2. Abstract or tangible concepts that are not readily available?
3. Can the answers to your questions be quantified (ratings/scores)?
4. Can entities at your level of analysis complete surveys?
5. Do you need data from a lot of potential respondents?

5x yes? -> Surveys are a good tool to collect data

Question 4

What is your population?

Answer 4

What kind of entities do you want to say something about?
• Which characteristics do they have?
• Which characteristics do they not have?
• How broad/narrow to focus?
• Generalizability vs. Specificity

Question 5

How to sample?

Answer 5

1. What kind
2. How to target
3. How many to target
4. What to offer in exchange

Question 6

What kind of respondents can help you answer the RQ?

Answer 6

• Representativeness

* Selection biases

Question 7

How to target respondents with those characteristics?

Answer 7

• Convenience vs. Random sampling

* Stratified sampling (comparing groups)

Question 8

How many to target?

Answer 8

• Response rates
• Non-response biases
• Response quality

Question 9

How many surveys do you need?

General rules of thumb

Answer 9

• 20 responses per variable (construct) collected by survey

* 30-50 per experimental condition

Question 10

How many surveys do you need?

Take into account…

Answer 10

1. Response Rate
   • FB/Twitter have very low response rates
   • Personalized e-mails have higher response rates than general e-mails.
   • Make sure to exert effort to increase the response rate (and decrease non-response bias)
2. Data Quality
   The more ‘committed’ an individual is to the project, the higher the data quality.

Question 11

Abstract constructs

Answer 11

• Use previously published/validated scales from literature
• Do not come up with your own questions (if needed adapt from other sources)
• Multiple questions for each construct (typically 3 or more)
• Use of Likert scales

Question 12

How many response options?

Answer 12

• Typically between 5 & 7 points
• Use original scale from the source
• Be consistent

Question 13

Tangible constructs

Answer 13

1. use single question
2. do not ask when you can collect from other sources
3. do not frame questions
4. do not ask questions you don’t need

Question 14

Open-ended questions

Answer 14

• E.g. asking respondents to state five challenging things in their job.
• Only if new/unknown input is needed

Question 15

Closed questions

Answer 15

• E.g. list 10 or 15 aspects that might seem challenging in jobs and ask the respondents to rank the first five among these.
• Closed questions help respondents make quick decisions.
• They help the researcher to code the information easily for analysis.

Question 16

A double-barreled question

Answer 16

A question that lends itself to different possible responses to its subparts

Such questions should be avoided and two or more separate questions asked instead.

Question 17

Non-neutral questions

Answer 17

Disagreeing can mean two things

• not very satisfied (but still satisfied)
• dissatisfied

Avoid loaded wording (e.g. awful, amazing)

Question 18

Social desirability tendency

Answer 18

Sometimes certain items that tap social desirability are deliberately introduced at various points in the questionnaire and an index of each individual’s social desirability tendency is calculated therefrom. This index is then applied to all other responses given by the individual in order to adjust for social desirability bias.