Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Intro to Social Research Methodology > Questionnaire Design > Flashcards

Questionnaire Design Flashcards

1
Q

Main Decisions when Designing Research Instruments

A
  • Type of Questions
  • Wording of the questions; how to ensure that they are clear; what are the possible answers
  • Design of the questionnaire (set the right order of questions and answers); check for the instructions
  • Wording of an accompanying letter
  • Method of distribution and return of the completed questionnaires (how the data collection mode limits the questions posed)
  • Methods for collecting the data; actions to be taken if questionnaires are not returned
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Question Content (3 things)

A
  1. Behavior
    * Want to know what the respondents are doing
    * When you interpret the findings, NEVER assume it’s a direct reflection of their behavior
    * Verbal report - they only think or say that they go twice per week
    * Theoretically we would have a chance to double check information
    * Response bias; social desirability
  2. Beliefs, Attitudes
    * How satisfied are you with
    * Cannot double-check; have to rely on responses
  3. Characteristics, Attributes
    * Gender, age, marital status, highest education, etc.
    * These questions aren’t part of your findings
    * Without demographic info, you can’t see the difference in attitudes between groups
    * It’s really easy to forget these demographic questions - don’t forget
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Types of Questions (4 types)

A

Direct vs. Indirect

Open-ended vs. Closed

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Direct Questions

A
  • Ask the respondent directly what’s their opinion; using “you”
  • What brand do you like?
  • Most questions should have a direct form
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Indirect Questions

A
  • Projective questions - qualitative
  • Which brand do people like?
  • When asking about sensitive information, indirect questions might be better
  • Must be careful when interpreting because answers are not straightforward
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Open-ended Questions

A

“Questions for which the respondent is asked to provide his or her own answers. In‐depth, qualitative interviewing relies almost exclusively on open‐ended questions. See Chapters 8 and 11.”

  • Allow respondents to present their own answers; no options
  • Give answers in their own words; use their own knowledge
  • More difficult to respond to these questions
  • Enables us to see how they formulated those answers
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

Closed Questions

A

“Survey questions in which the respondent is asked to select an answer from among a list provided by the re‐ searcher. Popular in survey research because they provide a greater uniformity of responses and are more easily processed than open-ended questions. See Chapter 8.”

  • Provide wording of question and answers for respondents to choose one
  • Yes/no questions; multiple choice
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

Open-ended Question Disadvantages

A
  • Many respondents skip these questions
  • If it’s part of an interview, the first response is “I don’t know”; researchers must probe of
  • More difficult for researchers to analyze; categorize answers; code them
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Types of Open-ended Questions (2 categories & 5 types)

A

Infinite responses
vs.
Limited amount of answers

  1. Starting questions
  2. Follow-ups
  3. Grand Tour
  4. Questions bringing arguments
  5. Questions bringing attitudes
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

Infinite responses

A

What do you think of our current president?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Limited amount of answers

A

What brand of yogurt do you get?

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

Starting questions

A

What role does TV have in your everyday life?

  • Gets people thinking about the topic; focus the respondent’s attention to the topic
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

Follow-up questions

A

What exactly do you remember from the advertisement?

  • Can distinguish liars; if they did see the ad, they could give something specific; if they didn’t see the ad, they’d give something general or incorrect
  • Without a follow-up, you’d get biased info
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Grand Tour questions

A

Please describe your usual activities during a workday?

  • Could give a lot or a little
  • Want to know how the respondent presents activities; how much info they can provide for the broad question
  • Want to see the emphasis they put on each part
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

Questions bringing arguments

A

Why did you purchase the product?

  • Makes person defend themself; Gives benefits of the product
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

Questions bringing attitudes

A

What do you think about the President?

  • Not about the object but the subject; asking about that person’s opinion
17
Q

Types of Closed Questions (6 types)

A
  1. Lists
  2. Categories
  3. Rankings
  4. Scales
  5. Quantities
  6. Grids
18
Q

List Questions

A

Which of the following research activities have you engaged in the last seven days?
If you haven’t engaged in a particular activity, please leave this box blank:
(check all that apply)

[ ] Accessed an online database
[ ] Accessed the internet
[ ] Read a journal article
[ ] Conducted an interview

  • Interviewers should read the question, then read each question, letting the respondent answer yes/no individually before moving onto the next item in the list
19
Q

Category Questions

A

In the last month, how often have you visited the library?
(choose only one option)

[ ] At least once a day
[ ] At least once a week
[ ] At least once every two weeks
[ ] Once within the last month
[ ] I have not visited the library in the last month
  • Easy to respond
  • Here, read all the response items and ask them to respond after
  • 3-5 options; more not recommended
  • Should create a “showcard” with all options that you can give/show to the respondent
20
Q

Ranking Questions

A

Please rank the following in order of importance to you in relation to your MSc studies. The most important should be given a rank of 1, the next most important rank of 2, etc. If you feel that a factor is of no importance, please leave it blank.

[ ] the course material
[ ] the library facilities
[ ] the computer facilities
[ ] the lecturing staff
[ ] assessment methods
  • People can usually identify most important but have trouble with later on the list
21
Q

Scale Questions

A 
XY has been an excellent Prime minister
1 - Strongly agree
2 - Agree
3 - Neither agree nor disagree
4 - Disagree 
5 Strongly disagree
  • Type of category question
22
Q

Quantity Questions

A

How many times have you been to the USA? ___
What year were you born? ___

  • Open-ended
  • Response is always a number
23
Q

Grid Questions

A

For your research work, how often do you use the following type of information source?

  • List of categories in the left column followed by a scale next to each item, for example 1 (Never) to 7 (Very frequently)
  • Saves space and time on questionnaires
  • Avoids repetition of (almost) the same question for multiple items
24
Q

Common Mistakes in Questionnaires (6 types)

A
  1. Double barrel
  2. Generalizing
  3. Mismatched wording
  4. Branching
  5. Mutual exclusivity
  6. Variability
  7. Uncertain options
25
Q

Double barrel mistake

A

Question:
How many cups of coffee or tea do you drink in a day?

  • Not objective - how big is your cup?
  • May not drink coffee or tea
  • May drink more/less on different days
  • There are coffee drinkers and tea drinkers

Principle:

  • Ask for an answer in only one dimension
  • Double barrel question

Solution:
* Should be separated into two different questions:

  1. How many cups of coffee do you drink in a day?
  2. How many cups of tea do you drink in a day?
26
Q

Generalizing mistake

A

Question:
What brand of computer do you own?
a. IBM PC
b. Apple

  • Missing lots of brands
  • May not own a computer

Principle:

  • Avoid hidden assumptions
  • Make sure to accommodate all possible answers

Solution:

  1. Make each response a separate dichotomous item
    a. Do you own an IBM PC? (yes/no)
    b. Do you own an Apple computer? (yes/no)
  2. Add necessary response categories and allow for multiple responses

a. What brand of computer do you own? (circle all that apply)
* IBM PC
* Apple
* Other

27
Q

Mismatched wording mistake

A 
Question: 
Have you had pain in the last week?
[ ] Never
[ ] Seldom
[ ] Often
[ ] Very often

Principle:
Make sure questions and answer options match

Solution:
Reword either question or answer to match

How often have you had pain in the last week?
[ ] Never
[ ] Seldom
[ ] Often
[ ] Very often
28
Q

Branching mistake

A

Question:

  1. Do you currently have a life insurance policy? (Yes or No; If no, go to question 3)
  2. How much is your annual life insurance premium?

Principle:
Avoid branching to avoid confusing respondents

Solution:
Write as one question
1. How much did you spend last year for life insurance? (Write 0 if none)

  • If people see repetitive branching in questions, people will adjust their answers to avoid the pain in the ass of answering the confusing questions
  • Speeds up interview
29
Q

Mutual exclusivity mistake

A 
Question: 
Where did you grow up?
a. Country 
b. Farm
c. City

Principle:
Avoid questions having non-mutually exclusive answers

Solution:
Design the question with mutually exclusive options

Where did you grow up?

a. House in the country
b. House in the city
c. Farm in the country

30
Q

Variability mistake

A

Question:
Are you against drug abuse?

Principle:
Write questions that will produce variability in the responses

Solution:
Eliminate the question

  • Social desirability effect - nobody supports drug abuse
  • Usually used to prove a point in data collection
31
Q

Uncertain options mistake

A 
Question: 
Which one of the following do you think increases a person’s chance of having a heart attack the most? (Check one)
[ ] Smoking
[ ] Being overweight
[ ] Stress

Principle:
Encourage the respondent to consider each possible response to avoid the uncertainty of whether a missing item may represent either an answer that does not apply or an overlooked item.

Solution:
Which of the following increase the chance of having a heart attack?

Smoking:
[ ] Yes
[ ] No
[ ] Don’t know

Being overweight:
[ ] Yes
[ ] No
[ ] Don’t know

Stress:
[ ] Yes
[ ] No
[ ] Don’t know

  • Don’t know is an important piece of information that you wouldn’t get from “check all that apply” instruction from the first question; distinguishing “yes/no” from “don’t know”
32
Q

Mistakes overview

A
  • One question may have many types of mistakes
  • Some questions were taken from real questionnaires
  • Some things are wrong every time; some items are mistaken only based on the context of the rest of the questionnaire
  • You can reuse questions from a questionnaire, but be careful - it could make sense and be a good question based on the context there, but it might be a mistake in the context of your own
    • How many cups of coffee or tea do you drink in a day?
    • Originally taken from questionnaire trying to assess consumption of cold drinks vs. hot drinks
33
Q

Pilot Study Benefits

A
  • Allows problems in questionnaire to be identified & corrected
  • Predict how respondents will interpret & react to questions
  • Opportunity to refine & develop interviewing skills
  • Highlight possible sources of interviewer bias
  • Provides feedback on respondents’ interest in the topic
  • Identifies the mistakes in instructions
34
Q

Impact of Numerical Labels

A
  • People attend to incidental features of questions, such as numerical labels
    • Responding on a scale of -5 to 5 vs a scale of 0 to 10
    • Uncommon scale can skew answers since people have to think much harder
    • May assert negative connotation on lower scores
35
Q

Sources of Response Bias

A
  • How the order in which the alternatives are listed affects the distribution of replies
  • Slightly higher responses for items that are ordered first
Much greater (5%)
Somewhat greater (9%)
About the same (48%)
Somewhat less (26%)
Much less (12%)

VS.

Much less (19%)
Somewhat less (23%)
About the same (46%)
Somewhat greater (10%)
Much greater (2%)
36
Q

Face vs. Radio Button

A
  • NPS Scores
  • Overall, how do you rate [journal name]?
Fixed (smileys):
1 - very good - 12.1%
2 - good - 36.4%
3 - satisfactory - 36.4%
4 - adequate - 10.1%
5 - inadequate - 4%
6 - unsatisfactory - 1%
Radio buttons:
1 - very good - 4.6%
2 - good - 52.8%
3 - satisfactory - 31.3%
4 - adequate - 7%
5 - inadequate - 3.2%
6 - unsatisfactory - 1.1%
  • bigger distribution with smileys perhaps because of the images
37
Q

Context and Images

A
  • Images added to illustrate questions may have unintended effects on interpretation of question
  • Images are necessarily concrete and may affect construal of category
  • Images are powerful contextual stimuli

Example:
How often have you been shopping in the last month?

(Picture of grocery store)
vs.
(Picture of clothing store)

38
Q

Before you start a questionnaire

A

Be very clear about what you need to learn

  • What are the questions you have about your subject of study?
  • What questions emerge from your objectives?
39
Q

Checklist for Eliminating Questions

  1. Does the question measure some aspect of the research problem?
  2. Does the question provide info needed in conjunction with some other variable?
  3. Will most respondents understand the question? In the same way?
  4. Will most respondents have the information to answer the question?
  5. Is the instruction correct?
  6. Will most respondents be willing to answer it?
  7. Is other information needed to analyze this question?
  8. Should this question be asked of all respondents or only a subset?
A
  1. if NO revise or drop; if YES retain
  2. if NO, drop question; if YES, should one or both remain?
  3. if NO revise or drop; if YES retain
  4. if NO drop; if YES retain
  5. If NOT make it right; if YES retain
  6. if NO drop; if YES, retain
  7. If NO retain; if YES retain only if other info can be obtained
  8. if ALL retain; if only SUBSET retain if identifiable beforehand

Intro to Social Research Methodology (9 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions