Lecture 5: Survey Research Flashcards
What is the formal term for surveys?
Sample survey
What is a survey?
A method of collecting primary data in which info is gathered:
- from a representative sample of people
- using spoken or written questions.
Are surveys qualitative or quantitative?
Can be both. Qualitative - e.g. open ended questions Quantitative - e.g. scale items - Standardised - Pre-arranged order - Pre-coded fixed alternatives
What can survey data relate to? (7 main ideas)
- Demographic info (e.g. age, ethnicity, income)
- Factual info (e.g. product ownership)
- Social interactions (e.g. social networks, social activities)
- Cognition (e.g. beliefs in claims, memory for promotions)
- Affect (e.g. attitudes and opinions, preferences)
- Behaviour (e.g. purchase behaviour and purchase intentions)
- State and trait orientations (e.g. personality characteristics)
- and anything else that might be of interest to a market researcher
What are the advantages to surveys? (4 main points, 5 dot points)
Standardisation
- Established items (e.g. scales)
- Commonly understood instrument
Easily administered
- Large samples
- Low cost
- Geographical flexibility
Ease of data analysis
Generalisability of findings
What are the drawbacks of surveys?
Issues of poor design
- Validity
- Reliability
- Language (jargon, misinterpretation)
- Choices of scales
Issues of execution
- Interviewers (e.g. trustworthiness)
- Representativeness and size of sample
- Appropriate data screening, cleaning and analysis
What are the challenges of surveys? (6 points)
- People often base responses on recent occurrences or salient events
- People often lack insight into their own behaviours and experiences, and sometimes improvise
- People often respond in a socially desirable manner, or with what they feel is the “correct” answer
- People can interpret the same wording differently
- Asking participants the same question twice can yield different answers across sessions (and also within one session)
- Valences associated with words can lead responses, as can the intensity of the words
Unlike other research methods, surveys allow for…? (4 points)
- Ready critique from others
- Replicability
- Storage of raw data
- Alternative-analysis and re-analysis
Inaccuracies in survey data may be due to…?
- Reliance on data from sample rather than population (random sampling error)
- Imperfect design and data collection methods (systemic error/bias)
What are random sampling errors? (aka sampling errors)
Error associated with measuring a sample rather than the full population.
Chance variation in the sample (even randomly selected)
- sample size
- sample selection
What is systemic error/bias?
Non-sampling error, it’s an error resulting from some imperfect aspect of the research design, or in execution of the research
What is non-response error?
- Non-participants: who did not participate? Any systematic non-participation? Not at home, refusals.
- Self-selection bias: who did participate? Over-represents extreme positions, under-represents indifference. Check sample against wider population. Check sample against non-participants.
- Item non-response
What is self-selection bias?
When you decide if you participate in the survey, like if you choose to fill out a customer satisfaction form.
Over represents extreme positions, under represents indifferences.
Should be checked against wider population and non-participants
What are the types of response bias? (7 points)
- Acquiescence bias: tendency to respond positively to all questions or to consistently concur with a particular perspective
- Extremity bias: tendency to respond using extremes
- avoiding mid-point
- Interviewer bias: tendency to respond in a particular way because of the presence of the interviewer (or interviewer characteristics)
- Auspices bias: a tendency to respond in a particular way because of the organisation conducting the study
- Social desirability bias: tendency to respond in a way that shows the respondent in a socially desirable manner
- education, income, open-mindedness
- checks built into many surveys (e.g. grooming, etiquette)
- Deliberate falsification
- Careless responses
What are the types of administrative error?
- Data processing error: for example, data entry error
- Sample selection error
- error caused by improper selection of sample design or execution
- for example, inclusion of mid-level sports fans in sample, when goal is to recruit high-level sports fans
- Interviewer error: for example, interviewer mis-recording responses, or making assumptions, bypassing items
- Interviewer cheating: for example, falsifying data, skipping questions, or completing responses (finish task earlier, fulfil quota)
Time in surveys can either be…?
Cross-sectional
- Data collected at a single point in time
- Snapshot or slice analogy
- Various characteristics are measured
- Cannot show changes across time and can lead to misrepresentations. Consider internet usage statistics - reduction as you age?
Longitudinal
- Data collected at multiple points in time, allowing comparison of changes across time
- Time series analogy
- Same participants vs different participants
- Cohort studies
- Risks Time 1 survey influencing Time 2 survey: consider self-reported study behaviour
- Risks Time 1 respondents differing from Time 2 respondents: large samples
What does it mean by cross-sectional?
- Data collected at a single point in time
- Snapshot or slice analogy
- Various characteristics are measured
- Cannot show changes across time and can lead to misrepresentations. Consider internet usage statistics - reduction as you age?
What does it mean by longitudinal?
- Data collected at multiple points in time, allowing comparison of changes across time
- Time series analogy
- Same participants vs different participants
- Cohort studies
- Risks Time 1 survey influencing Time 2 survey: consider self-reported study behaviour
- Risks Time 1 respondents differing from Time 2 respondents: large samples
What are some types of personal interviews (3)
- Door to door
- Mall intercept
- Captive audience settings
What are the types of phone interviews? (2)
- Landline
* Mobile
What are some types of self-administered questionnaires? (3)
- Internet
What are the pros of personal interviews? (7)
- High participation (relative to other methods)
- Ability to prompt and use visual aids
- Can be longer
- Ability to probe
- Completeness
- Opportunities for feedback, clarification
- Low cost (e.g. mall intercept)
What are the cons of personal interviews? (6)
- High cost (e.g. door to door)
- Difficult to access some respondents
- Poor representativeness
- Urban bias
- Participant cooperation
- Interviewer bias
Phone interviews can be either: (2)
Computer assisted telephone interviewing (CATI)
- Computer + person
Computer automated telephone system (CATS)
- Computer with no person - Often pre-arranged
What are the pros of phone interviews? (6)
- Fast data collection
- High geographic flexibility
- Ease of call back
- Low cost
- Simplified data collection
- Technology assisted
What are the cons of phone interviews? (5)
- Supervision of interviewers
- Participant misunderstanding
- Difficulty in gaining participant cooperation
- Lesser anonymity
- Dinner time calls
What are the pros of mail surveys? (6)
Low cost Geographic flexibility No interviewer influence Anonymity Self administered Ease of follow-up
What are the cons of mail surveys? (6)
Non-response Item non-response Low response rates Participant misunderstanding Inability to clarify Must be short
How do you increase mail survey response rates? (8)
Stamped return envelope
Professional-appearance of questionnaires
Cover letter
Monetary incentives or premiums
Interesting topics (can be initial distractor/filler items)
Follow-up letters
Advance notification
Survey sponsorship - auspices bias
What are the pros of email surveys? (4)
Ability to include links, images, real-time content
Faster response
Low cost
Geographic flexibility
What are the cons of email surveys? (3)
Span reputation
Compliance with legislation
Limited by respondent’s technology
- Renderability issues - Mail filters
What are the pros of internet surveys? (5)
Cost effective
Geographic flexibility
Speed of data collection
Interactive
- can limit number of items based on earlier responses
- Low item non-response - “cannot proceed”
What are the cons of internet surveys? (5)
Self-selection Respondent access Low cooperation Drop-out rate Representativeness
What are commercial panels?
Large mailing list or database of households or respondents who have agreed to participate in periodic mail surveys, email surveys, product tests, etc.
What are the pros of commercial panels? (3)
High response rate (can be guaranteed)
Suitable for longitudinal
Respondents have been pre-screened
What are the cons of commercial panels? (4)
Cost
Reliance on screening processes of panel org.
Sample representativeness
Falsification issues
