Module 8: Survey Methods Flashcards
Types of survey questions
- Open-ended
- Closed-ended
- Partially open-ended
- Rating scale
Open-ended questions
Questions for which participants formulate their own responses. The length of the answer can be controlled by the researcher by the amount of open space left.
o Pro: Allow for more variety in answers.
o Con: Are difficult to analyze, because data need to be coded or reduced in some manner.
Closed-ended questions
Questions for which participants choose from a limited number of alternatives. When writing closed-ended questions, researchers must make sure that the alternatives provided include all possible answers.
o Pro: Easy to analyze.
o Con: Limit the range of responses from the participants.
Partially open-ended questions
Closed-ended questions with an open-ended “Other” option. If none of the alternatives provided is appropriate, the respondent can mark Other and write a short explanation.
Rating scale
A numerical scale on which survey respondents indicate the direction and strength of their response.
o Pro: easy to convert the data to an ordinal or interval scale of measurement and proceed with statistical analysis.
E.g. Likert rating scale
Likert rating scale
A type of numerical rating scale developed by Rensis Likert in 1932. Uses statements rather than questions. For a neutral option, you always pick an odd number of choices. The middle one will then be the neutral option. However, if you want to force respondents to lean in one direction, you should use an even number
Misleading type of questions
- Loaded question
- Leading question
- Double-barreled question
Loaded question
A question that includes non-neutral or emotionally laden terms. “Do you believe radical extremists should be allowed to burn the American flag?” The phrase radical extremists load the question emotionally, conveying the opinion of the person who wrote the question.
Leading question
A question that sways the respondent to answer in a desired manner. “Most people agree that conserving energy is important do you agree?” The phrase Most people agree encourages the respondent to agree also.
Double-barreled question
A question that asks more than one thing. Double-barreled questions often include the word ‘and’ or ‘or’. For example, the following question is double-barreled: “Do you find using a cell phone to be convenient and time-saving?” This question should be divided into two separate items
Response bias
the tendency to consistently give the same answer to almost all of the items on a survey
How to minimize: word the questions so that a positive (or negative) response to every question would be unlikely
Criteria of arranging the questions
- Put related questions in subsets
- Place sensitive questions at the end
- Ask questions about events in the order they occurred
- Place demographic questions at the end
(Placing things at the end will prevent you from receiving no data at all when they decide to skip it)
4 types of ways to administer a survey
- Mail survey
- Telephone survey
- Personal interviews
- Total design method
Mail survey
A written survey that is self-administered
Pro:
- less sampling bias, since everyone can receive it
- eliminates interviewer bias
- Collect more sensitive data
- Less expensive
- More complete, because participants take as much time as they need
Con:
- No researcher available to answer questions
- Low return rate (25-30%), follow-up (50%)
Interviewer bias
the tendency for the person asking the questions to bias or influence the participants’ answers
Telephone survey
A survey in which the questions are read to participants over the telephone. A technique known as random-digit dialing (RDD), made it possible to reach all households, even those that were unlisted
Pro:
- less sampling bias (95% has phone)
- Respondents can ask clarification
- Researchers can ask follow-up questions
Con:
- People more willing to say “no”
- Houses have more than one phone (problematic for representative sample)
- More mobile phones nowadays
- Time consuming
- Costly
- Interviewer bias
- More socially desirable answers
Personal interviews
A survey in which the questions are asked face to face. This type of survey taking is very rare and difficult to achieve
Pro:
- Facial or bodily expressions are also recorded
- Participants devote more time
- Respondents can ask for clarification
Con:
- Interviewer bias
- Socially desirable answers
- Time and expense
- Lack of anonymity (participants may not feel comfortable answering)
- Decreased response rate, people are hesitant to open door for stranger
Socially desirable answers
Response that is given because participants believe it is deemed appropriate by society rather than because it truly reflects their own views or behaviors
Total design method
The recommended approach nowadays is multimodal, with a combination of mail, email/web-based, and telephone and uses multiple follow-up procedures
Pro: response rate 75-85%
Sampling techniques
- Probability sampling
- Random sample
- Stratified sampling
- Cluster sampling
- Nonprobability sampling
- Convenience sampling
- Quota sampling
Probability sampling
each member of the population has an equal likelihood of being selected to be part of the sample.
- Random sample
- Stratified sampling
- Cluster sampling
Random sampling
Random sampling is achieved through random selection, a method of generating a random sample in which each member of the population is equally likely to be chosen as part of the sample
Stratified sampling
A sampling technique designed to ensure that subgroups or strata are fairly represented. If the population consists of 70% women and 30% men, then we need to ensure that the sample is similarly proportioned.
Cluster sampling
a sampling technique in which clusters of participants that represent the population are used. This type is used when the population is too large for random sampling. As the name implies, cluster sampling involves using participants who are already part of a group, or cluster
E.g. if you were interested in surveying students at a large university where it might not be possible to use true random sampling, you might sample from classes that are required of all students at the university such as English composition.
Nonprobability sampling
a sampling technique in which the individual members of the population do not have an equal likelihood of being selected to be a member of the sample. Nonprobability sampling is typically used because it tends to be less expensive and generating samples is easier. Consists of:
- Convenience sampling
- Quota sampling
Convenience sampling (haphazard sampling)
involves getting participants wherever you can find them and normally wherever is convenient.
Quota sampling
involves ensuring that the sample is like the population on certain characteristics. However, even though we try to ensure similarity with the population on certain characteristics, we do not sample from the population randomly. We simply take participants wherever we find them, through whatever means is convenient.
Contingency question format (routing)
letting them answer a question and having a back-up question when they have answered it in a specific way
Questions to skip (routing)
If the participant has answered in a specific way they will receive information on which questions to answer next or maybe skip certain questions
Matrix question format
Give multiple questions with the same answers possible (something like a Likert scale)
