W3 measurement and questionnaire design

Question 1

Modes of surveys

Answer 1

Written (online)
Written (mail)
Telephone
Face to face

top to bottom increases rapport and data quality however also increases costs and interviewer bias

Question 2

What is a survey

Answer 2

• set of questions that is trying to measure an underlying construct/s of interest administered to a sample of participants
• a survey instrument is the actual device that you give to people, it contains all of the measured variables of interest

Question 3

Questions for conducting a survey

Answer 3

• what is/are the research questions
• what is the budget, monetary and time
• > cost is a big driver is research gathering data
• > online = cheapest, F2F = most expensive
• > cost is correlated fairly well with response rate, sample quality and participants motivation

Question 4

Written (online) approach of survey

Answer 4

• a cost effective way of recruitment
• samples seem just as representative as other modes
• but similar problems with mail
• responses may be more accurate with sensitive information (anonymity increased)

Question 5

Written (mail) approach of surveys

Answer 5

• Rapport, control and cost is lower

- Nonresponse and misunderstanding is higher

Question 6

Telephone approach of surveys

Answer 6

• more rapport and less nonresponse than written

- less chance of demand/interviewer bias than F2F

Question 7

F2F approach of surveys

Answer 7

• highest rapport, response, data quality
• on the downside, most effects of demand/interviewer bias
• most expensive

Question 8

Potential problems with survey data

Answer 8

• misunderstanding the question
• participants not having access to explicit cognitive representation of the item
• retrospective recall bias

Question 9

Potential problems with survey data

Answer 9

• misunderstanding the question
• participants not having access to explicit cognitive representation of the item
• retrospective recall bias
• social desirability
• demand characteristics
• response sets

Question 10

Survey instrument

Answer 10

• e.g the questionnaire, structured interview questions
• avoid double-barrelled, leading, loaded, unclear items
• pilot and pretest items
• always check if there exists measures used in previous (good quality) research

Question 11

The interviewers

Answer 11

• experience matters
• training
• procedures
• coding, meaning of codes, need to pilot these too

Question 12

Special features of the social science research context

Answer 12

• participants have minds and views of their own
• > they respond to their interpretation of what you ask not to your intentions
• participants may try to guess why we ask what we ask
• > may colour their responses generally sequencing important

Question 13

Problems in questions

Answer 13

• bias, no different from interviews and conversation
• leading (and loaded)
• double-barrelled
• many (most?) popular media polls are like this

Question 14

Mistakes to avoid

Answer 14

• avoid leading questions
• avoid questions wrapped in social desirability
• avoid double-barrelled questions
• avoid long questions
• avoid negations
• avoid irrelevant questions
• avoid poorly worded response options
• avoid big words
• avoid words and terms that may be misrepresented