Questionnaire Design Flashcards
Main Decisions when Designing Research Instruments
- Type of Questions
- Wording of the questions; how to ensure that they are clear; what are the possible answers
- Design of the questionnaire (set the right order of questions and answers); check for the instructions
- Wording of an accompanying letter
- Method of distribution and return of the completed questionnaires (how the data collection mode limits the questions posed)
- Methods for collecting the data; actions to be taken if questionnaires are not returned
Question Content (3 things)
- Behavior
* Want to know what the respondents are doing
* When you interpret the findings, NEVER assume it’s a direct reflection of their behavior
* Verbal report - they only think or say that they go twice per week
* Theoretically we would have a chance to double check information
* Response bias; social desirability - Beliefs, Attitudes
* How satisfied are you with
* Cannot double-check; have to rely on responses - Characteristics, Attributes
* Gender, age, marital status, highest education, etc.
* These questions aren’t part of your findings
* Without demographic info, you can’t see the difference in attitudes between groups
* It’s really easy to forget these demographic questions - don’t forget
Types of Questions (4 types)
Direct vs. Indirect
Open-ended vs. Closed
Direct Questions
- Ask the respondent directly what’s their opinion; using “you”
- What brand do you like?
- Most questions should have a direct form
Indirect Questions
- Projective questions - qualitative
- Which brand do people like?
- When asking about sensitive information, indirect questions might be better
- Must be careful when interpreting because answers are not straightforward
Open-ended Questions
“Questions for which the respondent is asked to provide his or her own answers. In‐depth, qualitative interviewing relies almost exclusively on open‐ended questions. See Chapters 8 and 11.”
- Allow respondents to present their own answers; no options
- Give answers in their own words; use their own knowledge
- More difficult to respond to these questions
- Enables us to see how they formulated those answers
Closed Questions
“Survey questions in which the respondent is asked to select an answer from among a list provided by the re‐ searcher. Popular in survey research because they provide a greater uniformity of responses and are more easily processed than open-ended questions. See Chapter 8.”
- Provide wording of question and answers for respondents to choose one
- Yes/no questions; multiple choice
Open-ended Question Disadvantages
- Many respondents skip these questions
- If it’s part of an interview, the first response is “I don’t know”; researchers must probe of
- More difficult for researchers to analyze; categorize answers; code them
Types of Open-ended Questions (2 categories & 5 types)
Infinite responses
vs.
Limited amount of answers
- Starting questions
- Follow-ups
- Grand Tour
- Questions bringing arguments
- Questions bringing attitudes
Infinite responses
What do you think of our current president?
Limited amount of answers
What brand of yogurt do you get?
Starting questions
What role does TV have in your everyday life?
- Gets people thinking about the topic; focus the respondent’s attention to the topic
Follow-up questions
What exactly do you remember from the advertisement?
- Can distinguish liars; if they did see the ad, they could give something specific; if they didn’t see the ad, they’d give something general or incorrect
- Without a follow-up, you’d get biased info
Grand Tour questions
Please describe your usual activities during a workday?
- Could give a lot or a little
- Want to know how the respondent presents activities; how much info they can provide for the broad question
- Want to see the emphasis they put on each part
Questions bringing arguments
Why did you purchase the product?
- Makes person defend themself; Gives benefits of the product
Questions bringing attitudes
What do you think about the President?
- Not about the object but the subject; asking about that person’s opinion
Types of Closed Questions (6 types)
- Lists
- Categories
- Rankings
- Scales
- Quantities
- Grids
List Questions
Which of the following research activities have you engaged in the last seven days?
If you haven’t engaged in a particular activity, please leave this box blank:
(check all that apply)
[ ] Accessed an online database
[ ] Accessed the internet
[ ] Read a journal article
[ ] Conducted an interview
- Interviewers should read the question, then read each question, letting the respondent answer yes/no individually before moving onto the next item in the list
Category Questions
In the last month, how often have you visited the library?
(choose only one option)
[ ] At least once a day [ ] At least once a week [ ] At least once every two weeks [ ] Once within the last month [ ] I have not visited the library in the last month
- Easy to respond
- Here, read all the response items and ask them to respond after
- 3-5 options; more not recommended
- Should create a “showcard” with all options that you can give/show to the respondent
Ranking Questions
Please rank the following in order of importance to you in relation to your MSc studies. The most important should be given a rank of 1, the next most important rank of 2, etc. If you feel that a factor is of no importance, please leave it blank.
[ ] the course material [ ] the library facilities [ ] the computer facilities [ ] the lecturing staff [ ] assessment methods
- People can usually identify most important but have trouble with later on the list
Scale Questions
XY has been an excellent Prime minister 1 - Strongly agree 2 - Agree 3 - Neither agree nor disagree 4 - Disagree 5 Strongly disagree
- Type of category question
Quantity Questions
How many times have you been to the USA? ___
What year were you born? ___
- Open-ended
- Response is always a number
Grid Questions
For your research work, how often do you use the following type of information source?
- List of categories in the left column followed by a scale next to each item, for example 1 (Never) to 7 (Very frequently)
- Saves space and time on questionnaires
- Avoids repetition of (almost) the same question for multiple items
Common Mistakes in Questionnaires (6 types)
- Double barrel
- Generalizing
- Mismatched wording
- Branching
- Mutual exclusivity
- Variability
- Uncertain options