L7 - Constructing Surveys Flashcards
List three recommendations for creating your own survey
Find existing scales (recommended) or develop your own
Simple, understandable questions
Determine sample and sampling strategy
Minimize dropout and response bias
Name the two sampling methods (Kroschnick, 1999)
Non-probability sampling method
Probability sampling method
Non-probability sampling method
- Sometime we don’t want to generalize to the entire population but rather describe or explore phenomenons (case study)
- Purposive sampling: typical representatives from a specific category
Probability sampling method
- Random sampling
- Stratified (based on characteristics)
- Multi-stage cluster (select clusters, then randomly select units within clusters)
Explain representative sampling and why it is essential (Kroschnick, 1999)
- Representativeness to a larger population is often the goal of research (cf. external validity)
- Statistical generalization: Probability theory is used to estimate the likelihood that the patterns observed in smaller groups (the sample) will hold in a larger group (the population)
- Randomly selected sample: Each individual should have equal chance to make it to the sample
How can you maximize your response rate? (Kroschnick, 1999)
Several strategies to increase rate • Anonymous/confidential • Incentives • Follow-up Also: • Explain reason for study • Professional look and feel • No language errros • Promise feedback • Time, less is more
How can we determine our sample size?
The formular used in the report
Explain the three types of pretesting (Kroschnick, 1999)
•Conventional pretesting
•Behavior coding: Used to note when respondents is deviating from script
o Misread questions or ask for more information, sounds unsure when starting the answer
•Cognitive pretesting: The respondents think aloud while answering
Explain the two diffrent kind of questions and one potential issue to each (Schaeffer and Presser, 2003)
Questions about events and behaviors: Typical questions ask about engaging in certain types of behavior/events.
Potential issues: Confusion of time and date of the events
Questions about subjective phenomena- attitudes (likert scale)
Potential issues: Forced to have an opinion
Explain uni- and bipolar scales (Schaeffer & Presser (2003)
- Unipolar- do not make assumptions about opposites
* Bipolar- make assumptions about opposites and have a neutral or ambivalent mid-point
Name pros and cons with the “dont know” option (Schaeffer & Presser, 2003)
- Allows for skipping questions –> Increase social desirability bias
- Allows for the respondents to not consider the question
- Good when the respondent do not have an opinion
What to avoid in a survey
- Leading questions
- Referencing authority
- Ambiguous questions
- Vague terms: Long time, Fairly, In general, Kind of
- Too specific terms
Name the four biases in surveys
1) Satisficing
2) Acquiescence
3) Social desirability bias
4) Common Method Variance bias (CMV)
1) Satisficing. Also explain what to do about it
When a respondent conserves energy by compromising the standards of answering the questionnaire and trying to complete it as fast as possible.
What to do about it
• Clear questions
• Avoid large batteries that are easily scammed
• Include attention check questions
2) Acquiescence. And what to do about it?
• A tendency to endorse any assertion made in a question, regardless of its content, perhaps due to human tendency to appear agreeable or due to human cognitive biases
o Ex.: I love dancing / I hate dancing- should have completely opposite responses, but they don’t, about 22% of respondents across studies would agree with both!
What to do about it?
Rephrase the question
