Exam #2 - Chps. 5-8 Flashcards
A set of structured questions to which a respondent replies is a…
questionnaire
Questionnaires provide ____________ and _____________ in data gathering.
standardization and uniformity
The questionnaire must translate the information needed into a set of specific questions that the respondent…
can and will answer
What is the questionnaire design process?
- Confirm research objectives and information requirements
- Select appropriate data collection method
- Develop questions and scaling
- Determine layout and evaluate questionnaire
- Obtain initial client approval
- Pretest, revise, and finalize questionnaire
- Implement the survey
What is a a double-barreled question?
two or more questions are combined into one
ex. “Do you think Coca-Cola is a tasty and refreshing soft drink?”
How can you increase the willingness of participants?
- Place sensitive topics at the end of the questionnaire.
- Preface the question with a statement that the behavior of interest is common.
- Ask the question using the third-person technique. Phrase the question as if it referred to other people.
- Hide the question in a group of other questions which respondents are willing to answer. The entire list of questions can then be asked
quickly. - Provide response categories rather than asking for specific figures.
- Use randomized techniques.
What are unstructured questions?
open-ended questions
respondents are asked to reply in their own words
What are advantages of unstructured questions?
useful for when you can’t come up with an exhaustive list of all of the choices
What are the disadvantages of unstructured questions?
might not get what you’re looking for
hard to analyze / get into numerical form
What are structured questions?
where respondents choose from a pre-specified list of answers
What are the three types of structured questions?
- dichotomous
- multiple choice
- scales
What are the advantages of structured questions?
easy to analyze
What are the disadvantages of structured questions?
takes more time to come up with an exhaustive list
might not capture all
What are dichotomous questions?
a question that has only two response alternatives: yes or no,
agree or disagree, etc.
For dichotomous questions, the two alternatives of interest are supplemented by a…
neutral alternative (no opinion, both, don’t know, etc.)
What are multiple choice questions?
the researcher provides a choice of answers and respondents are asked to select one or more of the alternatives given
True or False: Multiple choice questions often provide more clarity and refinement than dichotomous questions.
TRUE!
What are scaled response questions?
the choices are designed to capture the intensity of the respondent’s answers, such as attitudes or intentions
Creating a continuum upon which measured objects are located is known as…
scaling!
Assigning numbers or other symbols to characteristics of objects according to certain pre-specified rules is known as…
measurement!
What are the rules for assigning numbers during measurement?
- One-to-one correspondence between the numbers and the characteristics being measured.
- The rules for assigning numbers should be standardized and applied uniformly.
- Rules must not change over objects or time.
What are the primary scales of measurement?
- nominal scale
- ordinal scale
- interval scale
- ratio scale
___________ scales partition data into mutually exclusive and collectively exhaustive categories.
nominal
__________ scales are nominal scales that can order the data.
ordinal
__________ scales are ordinal scales with equal intervals between points to show relative amounts.
interval
__________ scales are are interval scales a meaningful zero point.
ratio
_________ scales allows magnitudes to be compared arithmetically.
- nominal
- ordinal
- interval
- ratio
ratio
This is an example of what type of scale question?
Are you satisfied with your present diet?
___Y___ N (where 1=y, and 2=n)
nominal
This is an example of what type of scale question?
Rank the following foods in order of preference (1=most preferred, 4=least)
___ Carrots
___ Cupcakes
___ Fish
___ Macaroni and Cheese
ordinal
This is an example of what type of scale question?
I eat 5 servings of vegetables a day.
Agree Disagree
1 2 3 4 5
interval
This is an example of what type of scale question?
Please allocate 100 points based on your preferences to the following foods:
___ Carrots
___ Cupcakes
___ Fish
___ Mac ‘n Cheese
100 Sum
ratio
What analyses can you perform on data that come from nominal scale questions?
basic analysis.
- frequencies
- counts
- %s
numbers don’t have distance!
What analyses can you perform on data that come from ordinal scale questions?
basic analysis.
- frequencies
- counts
- %s
numbers don’t have distance!
What analyses can you perform on data that come from interval scale questions?
distance has meaning, so you can take averages.
also basic analysis.
- frequencies
- counts
- %s
What analyses can you perform on data that come from ratio scale questions?
zero has meaning, it means nothing.
also basic analysis.
- frequencies
- counts
- %s
What are the two main types of scaling techniques?
comparative and non-comparative
What falls under comparative scales?
- paired comparison
- rank order
- constant sum
What falls under non-comparative scales?
- continuous rating scales
- itemized rating scales
- Likert
- semantic differential
- Stapel
In (comparative/non-comparative) scales, the stimulus objects are directly compared.
comparative
In (comparative/non-comparative) scales, the scale data must be interpreted in relative terms and have only ordinal or rank order properties.
comparative
In (comparative/non-comparative) scales, each object is scaled independently of the others in the stimulus set.
non-comparative
In (comparative/non-comparative) scales, the resulting data are generally assumed to be interval or ratio scaled.
non-comparative
The most widely used comparative scaling technique is…
paired comparison scaling.
In paired comparison scaling…
a respondent is presented with two objects and asked to select one
according to some criterion.
In rank order scaling…
respondents are presented with several objects simultaneously and asked to order or rank them according to some criterion.
True or False: In rank order scaling, the options are always put in alphabetical order so there is no bias.
TRUE!
In constant sum scaling…
respondents allocate a constant sum of units, such as 100 points, to attributes of a product to reflect their importance.
What does giving something a ‘zero’ in constant sum scaling mean?
it’s unimportant
How do you know if something is twice as important as something else in constant sum scaling?
It receives twice as many points.
In continuous rating scales…
respondents rate the objects by placing a mark at the appropriate position on a line that runs from one extreme of the criterion variable to the other.
In itemized rating scales…
respondents are provided with a scale that has a number or brief description associated with each category.
How are the categories ordered in itemized rating scales?
by scale position
What are the three main examples of itemized rating scales?
- the Likert scale
- the semantic differential
- the Stapel scale
In this type of itemized rating scale, the respondents are required to indicate a degree of agreement or
disagreement with each of a series of statements about the stimulus objects.
the Likert scale
In this type of itemized rating scale, respondents are provided a seven-point rating scale with endpoints associated with opposite, worded labels.
semantic differential
True or False: Negative labels are always on the left side of a semantic differential.
FALSE! It switches intentionally so that respondents are forced to pay attention.
What are the two ways that items on a semantic differential scale may be scored?
+3 or -3
1-7 scale
In this type of itemized rating scale, the respondent is provided a unipolar rating scale with ten categories numbered from -5 to +5,
without a neutral point (zero). This scale is usually presented vertically.
the Stapel scale
What issues must you consider when designing itemized scales?
- Number of Scale categories
- Odd or Even # of Scale categories
- Balanced vs. Unbalanced Scales
- Forced vs. Nonforced scales
- Nature and degree of verbal description
- Physical form or configuration
If a scale is reliable, it is free of what kind of error?
random error
If a scale is valid, it is free of what kind of error?
measurement error
____________ means that a measure provides consistent results over time.
reliability
____________ means that a measure measures what it’s supposed to.
validity
True or False: A measure must be reliable in order to be valid.
TRUE!
These questions determine if a
respondent is eligible to participate
in the survey.
Qualifying/Screening Questions
These questions break the ice and
put the respondent at ease.
Introductory Questions/warm-ups
These questions focus on the
survey topic and reassure the
respondent that survey is easy.
Main Questions: Easy
These questions obtain the rest
of the information needed.
Main questions: More difficult
These questions obtain
personality related information.
Psychographics/Lifestyles
These questions classify the respondents.
Demographics
These questions identify the respondent.
Identification Information
A systematic process of recording the behavioral pattern of people, objects, occurrences, etc. without questioning or communication with the subject being observed.
observational research
The use of a questionnaire to gather facts, opinions, and attitudes is called…
survey research
What are some reasons to use observational research?
- better / more accurate track of purchase behavior
- gets rid of the accuracy bias, less pressure to give answers the interviewer wants
- dogs, babies (groups we can’t exactly survey)
- bathroom study at Atlanta Airport, bathroom attendant viewed
What are the four main study design choices in observational research?
- degree of structure
- degree of disguise
- setting
- method of administration
When looking at the study design choice of DEGREE OF STRUCTURE for observation research, what are the two main types?
structured and unstructured
When looking at the study design choice of DEGREE OF DISGUISE for observation research, what are the two main types?
disguised and undisguised
When looking at the study design choice of SETTING for observation research, what are the two main types?
natural and contrived
When looking at the study design choice of METHOD OF ADMINISTRATION for observation research, what are the two main types?
human and mechanical
For degree of structure, what is the difference between structured and unstructured?
structured - the researcher specifies in detail what is to be observed and how the measurements must be recorded
unstructured - observer monitors all aspects of the phenomenon that seem relevant to the problem at hand
For degree of disguise, what is the difference between disguised and undisguised?
disguised - respondents are unaware that they’re being observed; done with one-way mirrors, hidden cameras, etc.
undisguised - respondents are aware that they are under observation
For setting, what is the difference between natural and contrived?
natural - observing behavior as it takes place in the environment
contrived - artificial environment, like a test kitchen
For method of administration, what is the difference between human and mechanical?
human - actual people
mechanical - machines
What is a ‘mystery shopper?’
falls under [study design choices in observational research] -> [method of administration] -> [human]
checking in on employee behavior as a customer
What is a ‘one-way mirror?’
falls under [study design choices in observational research] -> [method of administration] -> [human]
behind a deli counter, for example
What are ‘shopper patterns?’
falls under [study design choices in observational research] -> [method of administration] -> [human]
grocery aisles, rearrange when people get comfortable
What is a ‘humanistic inquiry?’
falls under [study design choices in observational research] -> [method of administration] -> [human]
researcher becomes part of a community and observes from within
What is a ‘psychogalvanometer?’
falls under [study design choices in observational research] -> [method of administration] -> [mechanical]
measures skin response to seeing something
What are ‘eye cameras?’
falls under [study design choices in observational research] -> [method of administration] -> [mechanical]
eye tracking, where you look / what they’re doing
What are ‘response latencies?’
falls under [study design choices in observational research] -> [method of administration] -> [mechanical]
the longer you take to answer = the more uncertain w/ your answer
What is ‘voice pitch analysis?’
falls under [study design choices in observational research] -> [method of administration] -> [mechanical]
faster and higher when you’re excited
What is ‘brain wave research?’
falls under [study design choices in observational research] -> [method of administration] -> [mechanical]
measures brain waves
What is ‘scanner data?’
falls under [study design choices in observational research] -> [method of administration] -> [mechanical]
scans at grocery store
What is the difference between facts, opinions and attitudes?
facts - ethnicity, age, name
opinions - quality of a product / location, likes/dislikes
attitudes - how much you like something
What are some advantages to survey research?
- lots of info very quickly from a lot of people
- easy to analyze the data
- responses are more reliable
- takes out interviewer bias
What are some disadvantages to survey research?
- hard to find a lot of people willing to fill it out
- may not be able to know the extent of the options / personal context for each answer
- takes a lot of upfront work to design
- have to know the set of responses, can’t really expand on that
Go through the classification of survey methods.
TELEPHONE
- traditional
- mobile
- computer-assisted telephone interviewing
PERSONAL
- in-home
- mall intercept
- computer-assisted personal interviewing
SELF-ADMINISTERED
- mail
- mail panel
- mail interviews
- online
- email
- software (Google Forms, etc.)
- drop-off
When considering the control of a survey method, you consider…
- sample (can’t see the people responding)
- data collection environment (what they’re paying attention to)
- field force (controlling the interviewers)
What are some advantages of using a TELEPHONE survey?
Fast
High sample control
Good control of field force
Good response rate
Moderate cost
What are some advantages of using an IN-HOME survey?
Complex questions can be asked
Good for physical stimuli
Very good sample control
High quantity of data
Very good response rate
What are some advantages of using a MALL-INTERCEPT survey?
Complex questions can be asked
Very good for physical stimuli
Very good control of environment
Very good response rate
What are some advantages of using a CAPI survey?
Complex questions can be asked
Very good for physical stimuli
Very good control of environment
Very good response rate
Low potential for interviewer bias
What are some advantages of using a MAIL survey?
No field force problems
No interviewer bias
Moderate/High quantity of data
Social desirability is low
Low cost
What are some advantages of using a MAIL PANEL survey?
No field force problems
No interviewer bias
Low/moderate cost
High quantity of data
Good sample control
Low social desirability
Mail Panel
What are some advantages of using an ONLINE survey?
Visual appeal and interactivity
No interviewer bias
Low cost
Social desirability is low
Very high speed
Personalized, flexible questioning
Contact hard-to-reach
respondents
What are some disadvantages of using a TELEPHONE survey?
No use of physical stimuli
Limited to simple questions
Quantity of data is low
What are some disadvantages of using an IN-HOME survey?
Low control of field force
High social desirability
Potential for interviewer bias
Most expensive
May take longer
What are some disadvantages of using a MALL-INTERCEPT survey?
High social desirability
Potential for interviewer bias
Quantity of data is moderate
High cost
What are some disadvantages of using a CAPI survey?
High social desirability
Quantity of data is moderate
High cost
What are some disadvantages of using a MAIL survey?
Limited to simple questions
Low sample control for cold mail
No control of environment
Low response rate for cold mail
Low speed
What are some disadvantages of using a MAIL PANEL survey?
Limited to simple questions
Low/moderate speed
No control of environment
What are some disadvantages of using an ONLINE survey?
Moderate quantity of data
Low sample control
No control of environment
Low response rate
In primary data collection problems, what are questionnaire problems?
the language / gender / etc. norms may not translate
In primary data collection problems, what are sampling problems?
you may not be able to access the people you want to
In primary data collection problems, what is nonresponse?
women, stranger danger, fear of gov’t
A ____________ is the general target market, all potential buyers.
population
A ______________ is a subgroup of the population itself.
sample
A ____________ is when you talk to every member of a population.
census
__________________ results when the sample size selected is not perfectly representative of the population.
sampling error
How can you mitigate sampling error?
use a larger sample size!
SAMPLE OR CENSUS: If you have a small budget, you should go with a
sample
SAMPLE OR CENSUS: If you have a large budget, you should go with a
census
SAMPLE OR CENSUS: If you have a short amount of time available, you should go with a
sample
SAMPLE OR CENSUS: If you have a long amount of time available, you should go with a
census
SAMPLE OR CENSUS: If the size of your population is large, you should go with a
sample
SAMPLE OR CENSUS: If the size of your population is small, you should go with a
census
SAMPLE OR CENSUS: If there is a small variance in the characteristic, you should go with a
sample
SAMPLE OR CENSUS: If there is a large variance in the characteristic, you should go with a
census
SAMPLE OR CENSUS: If the cost of the sampling error is low, you should go with a
sample
SAMPLE OR CENSUS: If the cost of sampling error is high, you should go with a
census
SAMPLE OR CENSUS: If the cost of non-sampling errors are high, you should go with a
sample
SAMPLE OR CENSUS: If the cost of non-sampling errors are low, you should go with a
census
SAMPLE OR CENSUS: If the nature of measurement is destructive, you should go with a
sample
SAMPLE OR CENSUS: If the nature of measurement is nondestructive, you should go with a
census
SAMPLE OR CENSUS: If you should pay attention to individual cases, go with a
sample
SAMPLE OR CENSUS: If you should not pay attention to individual cases, go with a
census
What are the sampling method decisions?
- Bayesian vs. Traditional
- with replacement vs. without replacement
- probability vs. non-probability
What is the difference between Bayesian and traditional?
Bayesian - can make adjustments over time to get targets
Traditional - completely random, you get what you get
What is the difference between sampling with a replacement and without a replacement?
with a replacement - push repeatedly to respond
without a replacement - once you’re out, you can’t come back into the frame (contact once, oh well if they don’t respond)
What is the difference between probability sampling and non-probability sampling?
probability - every member has an equal, non-zero chance of being chosen
non-prob - some have a chance, some don’t
(Of the probability sample techniques), simple random is…
when you number everyone, like from 1-10.
(Of the probability sample techniques), systematic sampling is…
interval, you pick a random starting point in the list of people and then decide on the interval
example: number everyone, the start at #22 and skip by 3 each time so you interview #22, #25, #28…
(Of the probability sample techniques), stratified sampling is…
you split based on the frame and then sample within those lists
example: if you want 75% male and 25% female, you take that frame, split it, and then randomize within those lists
(Of the probability sample techniques), cluster sampling is…
when you randomly select clusters (South quad dorms, zipcodes) and do a survey within that cluster subset
Why is probability sampling good?
- margin of error can be computed (accuracy!)
- you can generalize results
Why is probability sampling bad?
- expensive
- complex in design
- takes time to execute
(Of the non-probability sample techniques), convenience sampling is…
choosing out of convenience, not everyone has a chance of being chosen
(Of the non-probability sample techniques), judgmental sampling is…
choosing people that you believe are representative of the population
(Of the non-probability sample techniques), quota sampling is…
find desired %s out of convenience
(Of the non-probability sample techniques), snowball sampling is…
when you let the respondents do the work for you, ask them to circulate the survey
What is an incidence rate?
the chance that the person you contact is in the target market
What is a completion rate?
how likely that people both start and finish the survey
PROB OR NON-PROB: If the nature of the research is exploratory, you should use
non-probability sampling
PROB OR NON-PROB: If the nature of the research is conclusive, you should use
probability sampling
PROB OR NON-PROB: If the nonsampling errors > the sample errors, you should use
non-probability sampling
PROB OR NON-PROB: If the sampling errors > the nonsampling errors, you should use
probability sampling
PROB OR NON-PROB: If the population is homogeneous / low variability, you should use
non-probability sampling
PROB OR NON-PROB: If the population is heterogeneous / high variability, you should use
probability sampling
PROB OR NON-PROB: If the statistical considerations are unfavorable, you should use
non-probability sampling
PROB OR NON-PROB: If the statistical considerations are favorable, you should use
probability sampling
PROB OR NON-PROB: If the operational considerations are favorable, you should use
non-probability sampling
PROB OR NON-PROB: If the operational considerations are unfavorable, you should use
probability sampling
