ch5 Flashcards
When to use surveys?
- When you want to say something about ap population, but cannot measure the whole population
- When you are interested in quantitative descriptors
- Personal measures
- Subjective measures
- When observations not possible/feasible
Survey research design decisions
- Operationalization of concepts
- Decide on survey mode
- Appearance of the questionnaire
- Data collection
- Operationalization
- Reduction of abstract concepts
* To render them measurable in a tangible way
Steps involved in operationalization
- Come up with a definition of the construct you want to measure
- Think about the content of a. measure, which is the instrument
- A response format, e.g. 7 Point Likert scale
- Reliability and validity of the assessment scale have to be assessed
When are single-item measures used
When concrete singular object/attritbute
e.g. What is your marital status? What is your profession?
Multi-item scales
- Use ‘off-the-shelf’ scales
* Develop your own scale
Advantages and disadvantages of off-the-shelf-scales
+ Known and “good” validity and reliability
+ Comparability of results
+ Low cost
+- Not tailored to your exact research need
- Requires translation if in different language (source of error)
What to avoid in developing questions
- Double-barreled questions
- Ambiguous questions
- Leading questions
- Loaded questions
- Double negatives
comparative scales (ranking scales)
used to tell preference between 2 or more items.
It is ordinal in nature
Paired comparison (comparative scales)
Respondent continuously picks between 2 options. Used to determine prefernces
Rank ordering (comparative scales)
enables respondents to rank brands
Constant sum (comparative scales)
e.g. diving 100 points among 5 brands
Non comparative scales (rating scales)
Each object is scaled independently of the other objects in the study
Continuous rating scale
A score is given to something.
E.g. rate the Bijenkorf on a scale of 0 to 100.
Likert scale
Agree/disagree on a 5- or 7-point scale. E.g.From strongly disagree to strongly agree.
Treated as interval.
Semantic differentials
Choice between 2 bi-polar options. Good/bad, modern/old fashioned.
Treated as interval.
Response categories for categorical scales (Nominal) should be…
- Mutually exclusive: only 1 answer applies
* Should be collectively exhaustive: the answer possibilities cover the realm of possible answers
Survey modes
How the data are collected.
Interviewer-assisted vs self-administered
Factors that play a role in deciding on survey mode
- Measurement: interactivity, multi-media, interviewer presence or self-administration
- Representation: coverage quality, sampling control, response rate
- Economics: sample size, questionnaire size, speed, cost
Mixed mode designs
•To trade off cost and error
e.g. using web surveys + mail surveys to senior citizens
Web: cheaper, mail: better coverage
Why pre-test your questionnaire
- Catch errors/unclarities
- Discover sensitive topics
- Check response categories
- Optimize length
How do you pretest your questionnaire
- Pick 5-10 people from target group
- Let them “think out loud” + observe
- Improve survey
Note: any testing is better than no testing
Response rate
of people that participated in the survey DIVIDED BY # of people sampled
How to increase response rate:
•Maximize rewards of participation
-Show appreciation
-Use interesting/friendly questionnaires
-Offer tangible rewards
•Minimize the cost of participation
-Minimize time and effort required
-Minimize the chances of feeling threatened by questions
•Maximize trust
-Ensure anonymity/confidentiality
-Open lines of communication with the participant
-Identify research with well-known legitimate organization
Validity
Does an instrument measure what it is supposed to measure?
Reliability
Are the data accurate (free from measurement error) and consistent (from one occasion to another)?
Measurement validity
- Provide precedence (in off-the-shelf scales, refer to other studies who used the same scales)
- Always provide sound logic to support that considerable conceptual overlap exists between measurement/proxy and construct
- Be aware: single-item measures for abstract constructs = low validity. Multi item is better!
Proxy
An indirect measure of the desired construct, which is strongly correlated to the construct. Proxy is commonly used when direct measures are unobservable/unavailable.
E.g. Fat -> BMI
Social desirability bias
Respondents may not always be willing to communicate their true response in case of sensitive issues
Topics such as: alcohol and tobacco, healthy eating, finances, taxes
To minimise socially desirable responding:
•Deliberately leading and/or loading the question to make the sensitive “normal”. e.g.:
- “everybody-does-it”
- “Assume-the behaviour”
- “Authorities recommend it”
- “Reasons-for-doing-it”
! Note: in all other instances, avoid leading and loading
Reliability of survey measures
- For multi-item measures: Cronbach’s alpha
- Cronbach’s alpha measures to what extent a st of items are inter-related
- High inter-relatedness = high reliability
- Cronbach’s alpha = k DIVIDED BY k -1 (sum of covariances / sum of variances and covariances)
- Cronbach’s alpha is between 0 and 1
- Values >.7 are considered acceptable
