Survey Flashcards
What is the process of questionnaire design?
- Decision about the survey content
- Decision about the question content
- Decision about the question format
- Decision about the question wording
- Decision about the question sequence
- Decision about the survey layout
- Pretest of the questionaire
What is the decision necessary in survey content?
Survey content: decision about the construct that are addressed in th survey
- Need to focus –>Optimal questionaire length
- Survey content also depends on target group
- Consumers
- Decision makers in companies
What are the techniques that can be used for the decision about the survey content?
Survey content:
- throrough familiaraization with the topic: What is already known
- Literature research: What control variables are commonly used?
- Prioriziation of research questions
On what does the survey content depend upon?
Survey content
Need to focus –>Optimal questionaire length
Survey content also depends on target group
- Consumers
- Decision makers in companies
What is the decision about question content about?
Question content: about the content of single questions/items
What are important decisions in the decision about the question content?
Important decision: use of new or existing scales
– For many phenomena there are already established scales (e.g., established scales/ tests)
– Acceptance of new questions only in the case of novel phenomena or the existence
of unreliable measuring instruments
– Often: established scales are extended by additional, newly developed items
Important decision: Number of questions
– One question when it is possible to precisely word the question
(e.g., regarding demographic information, numbers)
– Multiple questions (scale) when the risk of random error is important (e.g., attitudes,
opinions, organizational characteristics)
What is important to consider when considering invidiual items in a survey?
Deciding about items with respect to the target group
-> Established scales for measuring extraversion
Question content: what are guidelines for scale development?
Guidelines for scale development:
– Ideally: four to six items
– Use the same expressions for the same issues
– Point out the necessity of repeating similar questions
– The entire width of the phenomenon should be covered by items (try to achieve representativeness
What is the idea behind domain sampling?
central idea: Domain sampling:
For every phenomenon there is an infinite set of indicators
– Sampling theory can be applied to scale development (e.g., representativens)
– Remember:
Indicator resemblance (=high reliability) does not guarantee the
validity of measurement
What are the two formats possible in the decision about the question format?
open-ended vs. closed questions
Open-ended: respondents can respond in any way they want
Closed-questions: respondents choose an answer from a given list
What are the advantages of open-ended questions?
- participants can answer in their own way
- allows for unusual answers
- little prior knowledge of researcher necessary
What are the disadvantages of open-ended questions?
- quantification difficult
- larger effort on the part of respondent needed, especially in the case of written surveys
What are the advantages of closed questions?
- straightforward quantification
- simple to compare
- support in interpretation of questions
- allow for time-efficient question
“batteries”
What are the disadvantages of closed questions?
- loss of spontaneity
- completeness of answer categories often hard to achieve
What are the 3 objectives in question wording?
Three objectives:
- Simplicity
- Unambiguousness
- Neutrality
Question wording how achieving simplicity?
Rules for achieving simplicity:
– Short sentences (max. 15 words)
– No pyramiding (e.g., through relative clauses)
– Avoid negations
– Avoid foreign words, if possible
– Translate scientific or technical expressions in everyday language
– Computations should be done by the researcher!
– Avoid unnecessary variations in the format and structure of the questions
Question wording how achieving unambiguousness?
Instructions for achieving unambiguousness:
– Avoid universal expressions (e.g., “all”, “always”)
– Avoid vague quantifiers (e.g., “often”, “many”)
– Separate the definition and the question
– Clearly define focal concepts
– Clearly define the time span to which the question relates
– Refer to only one issue
– Use the same words for the same issue
Question wording how achieving neutrality?
Instructions for achieving neutrality:
- Avoid suggestive formulations
- No anticipation of answers (e.g., “Do you also agree?”)
- Avoid loaded words
Question sequence, what iis importat about the cognitive process in survey response?
Survey Response as Cognitive Process:
- Attitudes are developed instantly during survey responses.
- Early questions activate information for later questions (potentially)
- Involves assimilation and contrasting processes.
What is the typical effect of question sequence in survey about customer satisfaction?
Customer Satisfaction Surveys:
- Total satisfaction measured first leads to emotional dominance
- Measured last leads to cognitive dominance (depends on prior answers).
Order preference depends on product involvement:
- Low involvement: Measure satisfaction first.
- High involvement: Measure satisfaction last.
What os the solution in question sequence to avoid biases?
Solutions to Biases:
- Randomize question order.
- Use “buffer” questions strategically.
Carfully design the sequence:
- Simple interesting openers, from general to specific
- demographics at the end, sensitive questions later.
- For high relevance of “demand effects,” assess dependent variable before the independent variable.
Why is the question layout a critical sucess factor?
Layout of questionnaire as critical success factor:
- “should look as sharp as your résumé”
- Signals interest for the topic
- Signals researcher’s trustworthiness (important in the case of confidential questions)
- Facilitates completion of the questionnair
What are elements of a careful layout?
Elements of a careful layout:
– Accuracy
– Clear and simple design
– Clear identification of researcher and survey objective
– High-quality paper and print (if printed)
how to achieve the impression of short processing time?
Create the impression of short processing time:
– Do not order questions too densely
– Use readable letter size
– Clear instructions
– Clear visualization of the question flow
What is a pretest?
Pretest: Testing of the questionnaire with selected / representative members of the sample
What are the goals of a pretest?
Goals:
- Identification of ambiguous questions and problems of understanding
- Check for the completeness of answer categories
- Identification of potential information gaps among respondents
- Check the required time for completing the questionnaire
- First indication of validity and reliability of the measuremen
What are the methods of pretests?
Methods:
– Debriefing: discussion after answering
– Protocol method: discussion during answering
– Think-aloud method: participants express all their thoughts while filling out the questionnaire
–>No best method
What are the two methods of survey data collection?
Population of interest:
1.Census (complete population
2.Sample (out of the population)
What are the two sampling methods?
1.Nonprobability sampling
2.Random selection (probability sampling)
What are the different methods of random selection (probability samplng)
- Simple random sampling: randomly select 10% out of all
- Stratified sampling: divide in two groups (A) and (B) and select 10% from each
- Cluster sampling: Create clusters consisting of A and B, that differ between
- Multi-stage sampling: combination of different methods
What are the methods of Nonprobability sampling?
- Convenience sampling: contact people most convenient (family, friends)
- Quota sampling: goal reach quota=50%, quota needs to be achieved –>difference to straitified)
- Snowball sampling: ask suitable people to spread the survey wiithin thier network
- Sampling based on typicality: specifc targets are selectes, select individually who participates
