Surveys Flashcards
What are surveys
A method of gathering data from large samples of individuals through their responses to a structured set of questions
Large-C method, large number of respondents, standardized questions.
What arent they good for?
- They cant study past events or people
- Less useful for inductive, questions need to be defined before data collection.
Sampling
- Want to say something about populations, however cant survey whole populations.
Probability sampling
In probability sampling, all members of the population have a known, nonzero chance of being selected into the sample
Law of large numbers: with random selection, sample means tend to increasingly approximate population means as the sample size increases ( samples likely to be representative)
- As sample size increases the sample is more likely to represent the population.
Non prob sampling
Probability-based samples are the ideal, but can be expensive, impractical, and are increasingly unachievable due to low response rates
Non-probability samples
Probability of selection of units into the sample is unknown
Population inference more complex, and less certain.
- Qouta, volunteer or snowball sampling are more common.
Coverage error
A complete population list may not be available
Instead, need to draw on sampling frame, which however, may not cover the entire population
E.g., not everyone is listed in the telephone book (or even has a phone)
Or: the electoral roll includes all registered voters, but it does not include eligible voters who are not registered
Coverage error = certain segments of a population are not part of the sampling frame
Do surveys measure true opinions or behaviours
They can but not always
Nominal question format
Binary: e.g., yes/no
Multiple choice: red/blue/yellow/green/other; Conservatives/Labour/LibDems/etc.
Ordinal format
Behavioral frequency: Never/Rarely/Sometimes/Often or All/Most/Some/A few/None
Ranking: Would you please say which one of these you, yourself, consider the most important? And which would be the next most important?
Quant/continuous
On a scale from 1-10
Good survey design?
Valida question measures what it is supposed to measure
Reliable: question provides consistent measurement
Social desirability bias
People may not answer truthfully if they want to present themselves in a good light. People over-report what is desirable and underreport undesirable behavior or attitudes. For example, not admitting racist views.
Acquiescence bias
Some respondents have a tendency to agree with everything. One response is to mix positive and negative statements on the same concept.
Recall problems:
people may not recall their past behavior or attitudes, or they may incorrectly remember. Hard to avoid in surveys; if a key interest, a behavioral (non-survey) measure is generally to be preferred.P
Straightlining and other forms of satisficing
Satisficing occurs when respondents put in minimal effort to understand and answer a survey question. One common form is straightlining: More technically called non-differentiation in ratings, straightlining happens when respondents lose their motivation to engage with a survey because they’re bored, don’t have the mental energy, or they find the survey too complex and demanding. They therefore always provide the same answer.
Leading questions
For example, if you ask Leading questions indicate to respondents that certain responses are more useful or acceptable to those asking the question. Generally to be avoided. (e.g., if you’re interested in people’s views on migration, do not do this: “There is too much immigration. Do you also agree that we should cut immigration?)
Wording effects
If the question is not clearly formulated, people will not be able to clearly answer it. If questions contain more than one stimulus, some people may respond to one stimulus and others to another. For example, if you ask “What do you think about Donald Trump and his policies”, it’s possible that people may view Donald Trump and his policies differently. Better to ask separate questions.
Simple language
Avoid unfamiliar jargon, such as academic concepts not known in general population. Avoid complicated wording
Double-barreling
Two questions at a time
List experiments
Unobtrusively measure sensitive attitudes (e.g., racial bias) in a way that allows respondents to remain truthful but also not explicitly reveal their sensitive attitudes
Present respondents with a list, how many apply?
In one group you include a sensitive item, in the control you dont.
How is survery data analysed
Through quantative methods
- Linear regression
- Statisitcal inference etc.
