Sample Surveys Flashcards
Target Population
The set of units from the population to be studied
Survey population
Frame is the list that allows us to access the survey
population
A complete non-overlapping list of the persons or objects in the population
The survey population is the population who can be
reached via this sampling frame
Sample
Selected from the sampling frame, the group from which measurements will be sought
Responding Sample
The group of people or objects that are successfully
measured
Data collection from the responding sample, e.g.
questionnaire
Objectives
research questions
Resources
money/ budget, constraints, people, alternative data sources
Population of interest
details of population (scope)
units, reference period, geographic coverage
Sampling frame
is a list which enables us to sample and
contact members of the population
list of units, extra information
does it correspond to target population?
examples: electronic or hard copy white pages random digit dialing frame is every 10 digit number society membership customer database Medicare members
Planning a Survey
Objectives Resources Population of interest Sampling frame data collection analysis plan
Sample Survey Process
Define research objectives Choose mode of collection and choose sampling frame Construct and pretest questionnaire Design and select sample Design and implement data collection Code nd edit data Make postsurvey adjustments Perform analysis
Units
what are the objects we want to sample and measure? -people -households -businesses (what definition?)
Scope
who is in and who is out
Reference period
e. g. permanent residents as of 1/1/2015?
e. g. businesses in operation as of 1/7/2016?
Over-coverage
People in the survey population not in the target population
Under-coverage
People in the target population not in the survey population
If `coverage rate’ is too low, then may change the target population to something more achievable
Coverage rate
What target population is feasible?
Collection method
e.g. if frame has phone numbers only, then must use a
phone survey etc
Sampling design
If we want to target our sample in some way (e.g. select more women than men), we can only do this using information on the frame!
Probability sampling
is preferable in most situations
everyone in the survey population has a chance of being selected
Simple random sampling
every sample is equally likely
N is the population size, n is the sample size
assign a random number (1, …, N) to every unit and take
the n units with the smallest values
there are many other designs - seek advice!
e.g. a sample of streets, take all residents
Sample Designs
Simple Random Sampling Convenience Sampling Systematic Sampling Cluster Sampling Stratified Sampling
Stratified sampling
simple random sample is taken from each strata the
population has been divided into (e.g. states)
Systematic sampling
the elements of a frame are ordered and then chosen
systematically (e.g. every 10th element)
Cluster sampling
the population is divided into clusters/groups (e.g. city
blocks) and a sample of the clusters is selected, then
sample all the units within each selected cluster
Multistage sampling
like cluster sampling, but elements within each cluster are randomly selected
Main goal in surveys is to
compare between subgroups, or to compare across time
Responding Sample
Is an attempt to survey all units in the sample.
Some will not be contactable
Some will refuse to participate
Some will be ineligible for the survey (over-coverage)
Response rate
is an important measure of quality
Non-respondents
may be systematically different to respondents, leading to substantial bias
Improving Response Rate
An introductory statement, stating the relevance of the survey and reassuring and true statements about confidentiality
Professional appearance & conduct of interviewer
Survey organisation has a good reputation
Good questionnaire/interview design
Survey not too long
Callbacks
Incentives
Collection method
Collection Methods
Mail
Telephone
Field Interview
Internet Survey
relatively cheap
response rates may be poor: may need 3 mail-based
follow-ups
Telephone
require a frame of telephone numbers
coverage imperfect
call-backs needed
Field Interview
face to face interview
more detailed information and complex sequencing are possible
Internet Survey
new area
obvious challenges are coverage and response rate
Prototype form
Like a draft form of the survey
Pilot Study
Visit a small sample of individuals from the population
Ask them to ll out the form as if received by mail
Observe, ask questions as they go (hesitations etc)
What did you take this to mean?
Never defend the form
Probe everything - introduction, explanatory notes, examples; even colour and font
Need to secure their commitment
Professional appearance
clear, appealing and uncluttered layout more important than the number of pages
Consistent
typeface and fonts so that questions, instructions and explanations and examples are clearly delineated
First Page must have…
why the survey is being conducted
believable and true statements about confidentiality
the due date, how the form is to be returned
title of survey and survey organisation
how long the form should take to fill in
Questionnaire
Do not mix too many different typefaces
Use boxes and boundaries to clearly define where things should be
Specificc instructions and examples next to the relevant question
Use dot points for instructions and explanations
The design of instructions and explanations is as important as the design of the questions
Consider how data entry is to be done
Language issues
Assume a reading age of about 12
Simple, non-technical language, e.g. help instead of
facilitate
Avoid negatives
Make the meaning clear
Provide examples if appropriate
Be specific: who, what, when, where
A question should be about only one topic
Questions should be short, but not at the expense of clarity
Questions can include memory aids such as examples and scene-setting, providing they do not lead the respondent
Do not use double-barreled questions
Ordering questions
screen people out of questions that do not apply to them with clear sequencing
make it as clear as possible what you want them to do
e.g. use tick one, tick one or more, write here etc
ask general questions first
use transition phrases when changing topics within the
questionnaire
Presentation
present the answer categories vertically
use booklet form for the questionnaire
For each question on the form ask yourself why you are asking the question
Sampling Error
Sample survey estimates differ (size) from the result that a complete enumeration
For a probability sample - the size but not direction can be indicated from standard error
Other sampling methods - not possible to estimate
Reduce sampling errors by using probability sampling
methods
Types of Errors: Non-sampling Errors
Coverage error Non-response error Interviewer error Instrument errors Mode of data collection Respondent error
Coverage Error
some units were not on the sampling frame
Non-response error
errors because some selected units could not be contacted or refused to provide the information
Interviewer error
interviewers may affect the responses
Instrument errors
errors or differences due to the way the questions are asked or the instructions are given
Mode of data collection
different answers to the same question may be obtained when using different modes
Respondent Error
errors due to faulty reporting by the respondent
Reducing Non-Sampling Error
Careful design of survey instrument
Training interviewers
Pretesting the survey
Input editing: Remove any apparently contradictory data items e.g. male but pregnant, employed but under 15, etc
Output editing: Integrity checks, unexpected results, e.g. prevalence of a condition much higher in one state than another
