SOC200 - Surveys (Chapter 8) Flashcards

1
Q

SURVEY RESEARCH

A

form of research in which data are collected through predetermined set of questions administered to a sample
Ideally, the sample comes from your population of interest
Observation and analysis occur together

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Main Purpose of Survey Research

A

To generalize findings from sample to pop of interest
Medium for data collection
Probability sampling – generalizable results
You are never gonna get a perfect sampling frame that includes all of target pop
Just make it as inclusive as possible
Will always have errors

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

When is Survey Research Appropriate?

A

for all 3 main types of research (descriptive, exploratory, explanatory)
Good for when researchers want to know something about pop that is too big to observe directly

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

When is Survey Research Appropriate?

A

Excellent for measuring attitudes + orientations in large pop (political polling)
When done right, some of most valuable tools for social inquiry

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

When is Survey Research Appropriate?

A

Just limited by types of questions you can include
Effective of getting accurate view of characteristics
Economic + efficient

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Western World is Suffering from Survey Fatigue!

A

Constantly bombarded with surveys + data collection
No time + energy
Drive in collecting through electronic habits without consent

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Qualities of a “Good” Survey: The appropriate use of questions and statements

A

“I drink hard liquor…”: Less than once a week, 1 to less than 5 times a week, 5 to less than 10 times a week, 10
times or more a week

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Qualities of a “Good” Survey: The appropriate use of questions and statements

A

don’t work well: drink hard liquor: awkward. Better as question
sometimes better as a question, sometimes as a statement
Obama question: too long

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Qualities of a “Good” Survey: The appropriate use of open-ended and closed- ended questions

A

How many times a week do you drink hard liquor? – ambiguous, ppl can interpret it in many ways
How many times a week do you drink hard liquor? - 1-5 times a week, 5-10 times a week, 10 times or more a week

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Qualities of a “Good” Survey: The appropriate use of open-ended and closed- ended questions

A

how many times a week do you drink hard liquor? – no N/A, answers not mutually exclusive
BETTER: In the last 7 days, how many drinks containing distilled alcohol have you consumed?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Qualities of a “Good” Survey: Clearly and unambiguously worded items

A

How many car repairs did you do last year?
Do they mean car repairs that I personally did, or that my mechanic did? Do they mean “last year” as in the calendar year, or do they mean in the last 365 days from today?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Qualities of a “Good” Survey: Clearly and unambiguously worded items

A

What is your current marital status?
I think it’s great. But my wife, well… Who knows what she’s thinking! What do they mean by marital status? Do they mean whether my marriage is happy, sad, or whether I’m currently single, divorced, or widowed?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Qualities of a “Good” Survey: No “double-barrelled questions”

A

How would you rate the quality of the instruction in SOC 200?
How happy are you with your phone provider’s rates and customer service?
PROBLEM: one question is asking you about two different things (rates and customer service) – can be misleading
can have diff views on both parts
if it contains “and” – asking 2 things

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Qualities of a “Good” Survey: No “double-barrelled questions”

A

Do you think universities should lower tuition rates in order to increase post-secondary enrolment?
PROBLEM: question still asking about two things (should universities lower tuition rates + asking if you think lowering rates will increase enrolment)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Qualities of a “Good” Survey: Asking questions that respondents are competent to answer

A

Approximately how many bottles of beer have you consumed in your life?
Do you think that China’s Three-Gorges Dam project will be beneficial for its environment?: who knows this?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Qualities of a “Good” Survey: Asking questions that respondents are competent to answer

A

How many times do you blink your eyes in a day?: no one can count that high
What was the first brand of toothpaste you started using?
Optimal recall periods: when is the best time to get these answers
Is it relevant?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Qualities of a “Good” Survey: The questions are relevant to the population of interest

A

What is your favourite meal using pork?: ppl with dietary restrictions – not appropriate
Do you think that China’s Three-Gorges Dam project will be beneficial for its economy?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

Qualities of a “Good” Survey: The questions are relevant to the population of interest

A

On a scale of 1 to 10, with 10 being the most challenging, how would you rate your ability to balance your work and family responsibilities?: depends on the age range, probably not for us
Which of the following newspapers do you read most frequently?: replaced by internet

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

Qualities of a “Good” Survey: Having respondents that are willing to answer

A

Some questions may be too sensitive/embarrassing for particular demographic to answer/answer honestly
One recent study found that while U.S. respondents skip sensitive questions, Asian respondents often discontinue survey entirely
Culture + race has also been linked to response styles:
Differences in extreme responding have been found between a variety of national + ethnic groups

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

Qualities of a “Good” Survey: Having respondents that are willing to answer

A

tendency of some respondents to agree with questions regardless of content, norm of agreeableness, varies significantly by culture
Dealt with through question wording; ensuring anonymity; providing item response choice that allows respondents to say “prefer not to answer”

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

Qualities of a “Good” Survey: Shorter items are better

A

The respondent should be able to read the item quickly, understand its intent, and provide an answer without too much study
Reduces survey fatigue
Minimizes the number of potentially misleading answers because the question was too complex for the respondent to understand

22
Q

Qualities of a “Good” Survey: Shorter items are better

A

May reduce level of non-response/number of uncompleted survey items
keep the items simple, questions clear
harder to do in practice rather than in theory
too short: not communicating enough of question, background info

23
Q

Qualities of a “Good” Survey: Questionnaires that avoid negatively worded items

A

Negation = easy misinterpretation
instructor did not respond to students’ comments during class: Strongly Disagree, Disagree….
might skip/be confused

24
Q

Qualities of a “Good” Survey: Questionnaires that avoid negatively worded items

A

-

25
Q

Qualities of a “Good” Survey: No leading questions, jargon, vague concepts, or biased terminology

A

More people own an iPhone than any other smartphone. Do you own an iPhone?
leading: marketing strategy – get on bandwagon
setting you up to answer untruly
may feel embarrased
social desirability bias

26
Q

Qualities of a “Good” Survey: No leading questions, jargon, vague concepts, or biased terminology

A

Indicate your level of agreement with the following statement: “My recent vacation was just peachy”: no colloqualisms
How many times have you ever wanted to hit your wife?: scare respondent into saying 0 times, assumes that only men hit spouse, assumes not relevant to unmarried ppl – makes too many assumptions

27
Q

Designing Effective Questionnaires

A

Questionnaire format is extremely important for ensuring high response rates and accurate answers
Questions should be spread out and uncluttered
No more than one question on a line
Cluttered questionnaires can:
Confuse the respondent and the interviewer

28
Q

Designing Effective Questionnaires

A

simple questionnaire format will mean positive experience for respondent
easy to follow
hard to code + input data set
internet survey: control flow of research

29
Q

Example of a Good and Bad Questionnaire Format

A

trial: matrix format - complicated, not clear to read, jumbled, arrows confusing
revised: improved navigation, nice shading, get away from matrix, easier to comprehend, nicer to look at

30
Q

Item Response Formats

A

Boxes/circles for checking, placed underneath the question + ordered vertically
3. Code #: for circling, placed underneath the question, and
way we understand + process survey need to account for: right to left for chinese/arabic

31
Q

Item Response Formats

A

ordered vertically, specifying code # to be inputed
interviews know # is circled, not crossed out
data entry person would know what # to do
can use scantron

32
Q

Types of Question Formats: 1. The Contingency Question

A

whether question is asked + answered depends on response to first question in the series
set off to side + enclosed in box + arrow leads to next question
helps to put instructions at top of page to remind ppl to answer questions that apply to them
computer based: next question appears automatically

33
Q

Types of Question Formats: 2. The Matrix Question

A

ask several questions that have same set of answer categories
Uses questionnaire space efficiently
Faster for respondents to complete
Allows respondents to easily compare answers with earlier questions
response set: when you see this, they rip right through it without much though
statements must be short + clear

34
Q

Importance of Question Order

A

order of questions can have a substantial impact on:
overall survey response rate
# + type of questions answered
accuracy of provided responses
sensitive questions at beginning discouraging
demographic data at beginning – routine form – no motivation

35
Q

Importance of Question Order

A

more interesting questions at beginning so they glance + want to do it
respondents aim to be consistent with preceeding responses
randomized order not practical - unorganized
just be sensitive to effects of ordering
gaining rapport – start with non threatening common questions then move on to sensitive stuff later on
self administered: sensitive stuff later on

36
Q

Pretest

A

Imagine spending hundreds of thousands or millions of dollars on survey design, only to discover that poor response rates and misleading answers were caused by:
ambiguously worded questions and item responses
format of questionnaire + ordering of questions
survey that was not relevant for the intended sample

37
Q

Pretest

A

See how ppl respond
Did they give answers you weren’t expecting
Important they understand what you’re doing
Work out the bugs

38
Q

Methods of Collecting Survey Data: 1. Self-administered Questionnaires

A

Hard copies delivered in person/through mail/soft copies administered online via links to a website
monitor returns + ensure anonymity
graph – detect response bias, when, where, answers
properly timed follow ups – 2-3 week gaps
online questionaires similar: access those with internet – may not be generalizable

39
Q

Methods of Collecting Survey Data: Live Interviews

A

via face-to-face/live interviewing by telephone
demeanour: composure, language
how they dress + act must not influence responses
high response rate – harder to say no to someone
able to help fill out questionnaire – provide clarification
probe: minimize idk + prefer not to answer
not following script, dress (inappropriate, unprofessional) – bias/wilingness to participate
feel more guarded when talking to person
limit to what they can ask – may be untruthful

40
Q

Methods of Collecting Survey Data: Telephone

A

Via computer assisted telephone interviewing (CATI)
telephones used to be a luxury – built in bias
reaching respondents from certain class – answers reflect bias
access widened – representativeness increased
dropping landlines for cellphones, caller ID
nonresponse bias – less representativeness

41
Q

Example of a Survey Response Rate Graph

A

week 1: high response rate
drops off
follow up: spikes up again
those inclined to response = those who would reelect rob ford
track reaction to diff news stories – spike in disapproving responses who saw the news
track biases + reactions to events

42
Q

Interview Administered Questionnaire

A

Demeanour, familiarity with questionnaire + following question wording exactly key to successful interview administered questionnaire

43
Q

Telephone Surveys

A

Computer Assisted Telephone Surveys (CATI)

44
Q

Self-administered Questionnaires

A

Cheaper & quicker than face-to-face interviews
Require small staff
Higher response rate to sensitive questions
Makes large samples feasible

45
Q

Self-administered Questionnaires

A

Lower response rates

May have to limit survey to questions minimally appropriate for all respondents

46
Q

Interview-based surveys

A

More effective for complicated surveys
Can conduct survey based on sample of addresses/phone numbers
Face-to-face interviews allow relevant additional observations
Better at dealing with context of social life

47
Q

Interview-based surveys

A

More expensive, especially for national survey

Training + quality assurance of interviewers adds to reliability issues

48
Q

Telephone surveys

A

Saves time + money, relatively far-reaching for cost
Less risk to interviewer’s safety
Impact of interviewer on responses is lessened

49
Q

Telephone surveys

A

Easy for respondents to ignore, increasing non-response bias

interviews: lower response rates to sensitive questions

50
Q

Online surveys

A

Same strengths as self-administered questionnaires, but cheaper

51
Q

Online surveys

A

No way to ensure representativeness of some general pop

May be impersonal + thus easily ignored, leading to lower response rates