M6 - Survey design Flashcards
Survey design problem
You cannot talk
–> must be clear; no clarification possible
–> build trust ; ask for sensitive info
–> common-method bias
People tend to tick the answer again
Survey design elements needed
Where to put them?
Welcome - set the tone Legitimation - importance Motivation - prices Story - goal Trust - sensitive info asked
Where:
invitation letter / survey cover sheet
Instructions on page 1 of the survey
Actual questionnaire
Steps in a survey project
- research question & conversation
- pop/sample: who will we actually survey on?
–> the right tone/ customize language - first draft & feedback from
People not in the survey
–> length? Sense of questions? - pilot test with people from actual sample
–> is it clear? - actual survey
Concept
Abstractions communicated through words or other signs that refer to properties among phenomena
Conceptualization
Process to specify what exactly is meant by particular terms
Eg: attitued toward risk, leadership style
Variables
Characteristics of units, taking on diff values, categories, or attributes for diff observations
Eg: age, color
Types of variables
Dichotomous Discrete Continuous Manifest Latent
Dichotomous
Only 2 possible values
True/ not true
Discrete
A few possible values
AG/ gmbh/ uh
Continuous
Any value within a certain interval
Manifest
Directly observable
Traffic light color
An observable variable (manifest) is an indicator
Latent
Not directly observable
An intent to do sth
Indicator
What
Should be
A rule of correspondance that links a theoretical concept to observable facts
If a variable is directly observable (manifest), the variable is an indicator
Should be:
Unambiguous, understandable, precisely defined
Include all relevant aspects
Appeal to the respondents
Qualitative scales
Nominal (gender, name)
Ordinal (grade in an exam)
Quantitative scale
Interval (temp in degree)
Ratio
Absolute
Rating scales
Likert scale
Guttman scale
Respondent chooses the level that best represents his/her opinion
Likert scale
Measures the ….. of ….. to a given ……
Disadv:
Measures the degree of agreement on a statement
1-5
Disadv:
Potential bias to the middle
Guttman scale
Is a …. of ….. statements
A person who agrees to statement …, should also agree to statement … and ….
s a list of escalating statements
A person who agrees to statement 3, should also agree to statement 1 and 2
Semantic differential
Measures …. to …. words or statements on a … point scale with …. anchored in ….. adjectives
Semantic differential
Measures REACTIONS to STIMULUS measures words or statements on a 7 point scale with ENDPOINTS anchored in OPPOSITE adjectives
What is a scale?
Synonym - inventory
A number of questions/items that measure a construct
Scale like likert/guttman etc
A scale may habe suv-scales/dimensions
Whats KAI?
3 dimensions
Kirton-Adaption-Innovation Index
Measures how innovative one is
3dim:
How much individuals like to work with new ideas
How much individuals focus on being efficient
How much individuals confrom tu rules and authority
Reliability
What
Types of reliabiliyt
How well am i measuring what i want to measure?
Types: Test-retest rel. Parallel test rel. Split half rel. Inter-rate rel. Internal consistency
Types of reliability
TR PT Sh IR IC
Test-retest - same test at 2T
Parallel test - corr. Bet. Similar tests at 2T
Split-half: randomly divide sample in half
Inter-rate: for coding; agreement bet. coders
Internal consistency: corr. Bet. Items
How to increase validity?
Use existing scales
Pilot studies
Good questions
Multiple measurements
Where to find existing scales?
Research fields
Books
Hint 1 - apply minor tweak
Hint 2: use existing scale
Formulating surv y questions 10 rules
- avoid ambiguous wording
- avoid words or expressions the respondent may not know
- dont cover more than 1 content with 1 quest. Item
- avoid negative instructions
- avoid overly short / long sentences
- aboid complex questions
- avoid abbreviations
- aboid suggestive questions
- aboid moral obligation/ social desirability
- provide the full range of response options
Social desirability
Tendency to give answers of which the respondant thinks they are socially desirable or approved, rather than accurate and honest
Eg: are you sexist?
Acquiescence
Tendency to prefer positive over negative answers
Reduce throigh clear and unambigiuous questions
Closed questions
Questions phrased …
Answer options are ….
Open end questions are a ………
No ….. desired
Not …… defined
Closed questions
Questions phrased PRECISELY
Answer options are DEFINED
–> easier to analyse
Open end questions are a FREE TEXT FIELD
No PRIMING desired
Not PRECISELY defined
–> contain more info
Survey tools
Adv
Disadv
What to keep in mind?
+ customizable online programs
- manual entering as source of error
- data has to be processed to certain format for statistical software
When designing think about coding too!
Adv swlf-administered questionnaire
Cheaper Smaller number of skilled analyzers needed Fast analysis Large no. Of participants possible Anonymity & privacy
Disadv self administered questionnaire
Stop mid-way Skip answers Go through without answering No clarification possible Low response rate Might represent the most extremes of the pop.
Adv researcher administered questionnaire
Fewer misunderstanding, incomplete responses
Greater control over the environment
Clarification possible
Higher response rate
Disadv researcher adm. quest.
Expensive
Large number of skilled staff
Limited numbers
Long time for admin. & analysis
