Basic questionnaire design Flashcards
Define survey
A Survey is a general term for any kind of data collection on people’s opinions or behaviour.
E.g. A market researcher asking how you found the facilities at an airport.
Define questionnaire
A questionnaire is a set of questions (usually written) that people are asked to respond to – usually on a rating scale.
Thus, a questionnaire can be used as part of a survey
What people do, not how they think
What, how, when, how many…
What simple response categories can be used in questionnaires?
• Open ended: the person writes their answer.
• Which brand of toothpaste do you buy?
• How old are you?
• Category tick boxes: Are you aged 18-25, 26-35, etc.
• When should you have categories not actual replies?
• “Sensitive” data; Arbitrary banding; Unable to recover detail.
• Which of these news sources do you read? (tick all that
apply)
• How often do you read them? (daily, weekly, monthly).
• If the question needs them to choose only option from a
selection…
• Then just like your variables for Content Analysis the
categories must be exhaustive and mutually exclusive.
• The use of “Other” as a category.
• Transport to work example…all that apply? Or the longest
distince?
What value is used for a missing answer for questionnaires?
either 9 or mode
Ways to deliver a questionnaire?
telephone survey
researcher led questioning
paper based questionnaires
Online surveys
Telephone surveys pros and cons
• Can get to a wide variety of people geographically.
• How will you select your sample?
• Random number dialling?
• Not everyone has a landline or a mobile.
• And this is related to age – i.e. a biased sample.
• People are often annoyed by cold callers.
• It will rarely be a good time for them.
• You could pre-arrange a good time.
• With postal questionnaires, at least they can fill it in
when they choose. But they mainly forget or throw
it away.
• Telephone surveys are not used very often.
Researcher read questionnaire pros and cons
The researcher reads the questions to the participants and fills in the answers for them.
- Can be pre-arranged by appointment…
- Or opportunistic, e.g. asking people in a public place.
- Often used by market researchers on the street.
- This is the same as a structured interview with pre-determined response choices.
- The challenge is getting people to stop and take part.
- Useful if the respondent has difficulties with reading/writing.
Paper based questionnaires. pros and cons
• Paper based questionnaires. Can be:
• Handed out to people face to face, and:
• Completed there and then (e.g. in a class).
• Taken away and return by post or to a central point.
• Sent out in the post.
• The challenge is getting people to send it back.
• See response rates.
• You must report your response rates – i.e how
many you sent out compared to how many you
got back.
• Your response rates will be low.
Online Surveys pros and cos
• For example:
• Surveynet.ac.uk
• Smartsurvey.co.uk
• Surveymonkey.co.uk
• Onlinesurveys.ac.uk
• Here at Imperial students have access to:
• Qualtrics
• See Imperial website. Also gives tips on writing questions.
• Caution: This makes it seem very easy to produce and send out questionnaires.
• Fine for a survey but if you want a scientific product you need to be trained…these methods are what psychology students are doing for 3 or 4 years! I am giving you the basics of behavioural research methods.
• Even for a simple survey you still need Ethical approval
• The ethics lecture in SiC is part of this module and
examinable.
problems with internet and paper questionnaires
In common with postal questionnaires:
• People may ignore the inclusion criteria and fill it in even if they are not in the desired demographic category.
• Especially if there is an incentive such as a £5.00 Tesco voucher.
• Very easy to get bored or affronted and just hit exit.
• No knowledge of non completers (see later).
But also for internet questionnaires:
• Reach a selective (biased) population of users. * Okay if you are aiming for a young, relatively wealthy and educated sample. * On the positive side they are cheap and fast to distribute.
How to improve questionnaire response rates
• Don’t make the questionnaire too long.
• Four sides A4/40 questions max as a rule of thumb, but depends…
• It can be more if the participants are motivated in
some way. E.g. it will improve their own experience of something, e.g. their local area.
• Incentives help – tokens, entry into a prize draw.
• Explain why the results of the study are important (in your cover letter). More likely to give up their time.
• They need a prepaid return envelope if posted.
• Coloured paper is helpful in an office environment.
• Sending reminders – if you know who has received the questionnaire and it is not too expensive (e.g. email reminders).
What should you consider when conducting populations sampling measurements?
representative?
Random?
Convenience?
Systematically biased - self selected; accidental?
Chosen sub samples - women, commuters, students
Ask for demographic details or focus on the group only
What is volunteer and completion bias?
- Only a small proportion of people given questionnaires return them.
- If you send them out randomly then you only have demographic data on those who reply.
- We do not know anything about the people who didn’t respond.
- Those who answer may be different somehow – and in a systematic way.
- For example, those who volunteer to take part in research are typically more educated than the general population average.
Problems when the wrong people fill out the questionnaire
- Even though you specify who you want to complete the questionnaire in your introduction…
- You get people who were not in your inclusion criteria.
- This can happen with paper and online questionnaires.
- Some people add a question to check.
- For example, you only want male students and said so at the start.
- But now a whole load of women have filled it in and ticked female on the questionnaire.
- Did they not read the introduction?
- This will be particularly problematic if there is a financial incentive.
Wording questions for questionaires
• Keep questions short and simple.
Avoid question that are too general.
• For instance “Do you agree with current terrorist sentencing policy?”
• Covers much they may not know about, or they may agree with some but not all aspects.
Avoid implying the socially desirable answer or asking leading questions:
• Most people think single use plastics should be restricted, do you?
• Do you agree that terrorists released from prison are a serious danger to the community?
