Lecture 8 Flashcards
Surveys are valuable tools for ____ ____ and anyone wanting to collect original data to _____ a population that is …
- social scientists
- describe
- too large to observe directly
Prevalence of polls (and telemarketers) may have ______ the status of surveys in the eyes of the public.
diminished
Survey:
technique that seeks to determine present practices or opinions of a specified population
Questionnaire:
type of paper and pencil or electronic survey in which information is obtained by asking participants to respond to questions rather than by observing their behaviour
The purpose of survey design is to provide _____ or _____ description of _____, _____, or _____ of a population by studying a sample of that population.
- quantitative
- numeric
- trends
- attitudes
- opinions
Most common type of descriptive research:
survey research
Pros of survey research:
- Can solicit info from large populations.
- Best at determining current opinion or trends in a population.
- Can provide a basis for making broad based comparisons &
decisions
Cautions of survey research:
- must ensure the population is adequately represented
- reliant on the honesty of the respondent
- very difficult to establish cause and effect
- difficult to gain the underlying reason for the info solicited
5 methods of a survey method plan:
- the survey design
- the population and sample
- creating and pretesting an instrument
- variables in the study
- data analysis and interpretation
Survey design:
a plan for quantitative or numeric description of trends, attitudes, or opinions of a population by studying a sample of the population
In the survey design, we indicate why a survey is the ____ type of data collection procedure for the study, examine the _____ of this design for the study.
- preferred
- advantages
2 types of survey design:
- cross section
- longitudinal
Cross-section survey design:
participants assessed at one point in time
Longitudinal survey design:
following participants over time
In a survey design, we must specify the form of ____ _____, examine the _____ and ______ for the ones chosen.
- data collection
- strengths and weaknesses
Types of data collection:
- telephone
- drop-off
- internet surveys
A challenge of questionnaires is that you only get responses to ….
the questions that you ask (if you ask a poor question, you may not get the types of responses that you were expecting)
When posing questions, make sure that the question _____ to the respondent.
applies
When posing questions, make sure the question is ______ ______.
technically accurate
When posing questions, ask ____ _____ at a time.
one question
When posing questions, use _____ and _____ words.
- simple
- familiar
When posing questions, use ____ and ____ words to specify the concepts clearly.
- specific
- concrete
When posing questions, use ___ words as possible to pose the question.
few
When posing questions, use ____ sentences with ____ sentence structures.
- complete
- simple
When posing questions, make sure ___ means _____ and ____ means ____ (ie. avoid ____ _____).
- yes means yes
- no means no
- double negatives
When posing questions, be sure that the question specifies the ____ _____.
response task
In the population and sample stage, identify the ____ in the study, state the _____ of the population, means of _____ individuals in the population, and _____ of sample frames.
- population
- size
- identifying
- availability
In the population and sample stage, we need to specify if the sampling design will be:
- single stage
- multi stage
Single stage sampling design:
- there is access to the names of the people in the population
- they may be sampled directly
Multi stage sampling design:
will identify groups or organizations who will assist in the gathering the names of individuals who will be sampled
In the population and sample stage, we need to identify the _____ process for the sample.
selection
In the population and sample stage, we need to indicate if the population and subsequent sample will be _____ based on …
- stratified
- specific population characteristics
- eg. age, gender, income levels, education
In the population and sample stage, we need to discuss how sample will be _____ from available lists.
selected
In the population and sample stage, we need to indicate the ____ of people in the sample and the ____ used to compute this number.
- number
- procedures
In the population and sample stage, attention must be given to the _____ of the population that will be included in the:
- percentage/fraction
- sample
- margin of error
- confidence interval
- confidence level
- response rate
The ____ ____ will dictate the type of participants you need for the study.
research question
In the population and sample stage, we need to ensure sampling/participants are ______ for your chosen research design and methods.
appropriate
The greater the variability in your outcomes, the _____ sample size needed to find _____.
- larger
- significance
In the population and sample stage, we need to think about _____ considerations (_____), such as…
- practical
- feasibility
- time, money, resources
- participant availability
In creating and pretesting an instrument stage, we need to name the survey ____ used to collect data.
instrument
When using an existing instrument, describe the established ____ _____ from previous use.
validity scores
Validity scores in a survey identifies ….
whether the instrument is a good one to be used
3 components of validity scores;
- construct
- content
- criterion
Construct (validity scores):
how one translates the idea or construct into something real or concrete
Content (validity scores):
a check of the operationalization against the relevant content domain of the construct
Criterion (validity scores):
the validation of a measure based on its relationship to another independent measure as predicted by your theory of how the measures should behave
In creating and pretesting an instrument stage, we need to mention whether scores resulting from past use of the instrument demonstrate _____.
- reliability
- test-retest correlations
- consistency in test administration and scoring
In the pilot testing phase, survey should be tested with a sample as _____ to the sample used in the ____ ____ ______ as possible.
- similar
- full survey presentation
4 stage process for pilot testing:
- review by knowledgeable colleagues and analysts
- interview with potential survey respondents to evaluate each question
- conduct a small pilot test
- a final check from a small group of individuals with no role in survey development or revisions
The variables in the study phase is useful in the _____ section to relate the variables to ____ _____ and items on the _____.
- methods
- research questions
- instrument
The variables in the study phase allows the reader to easily determine how the _____ ______ connects to the variables and _____ or _____.
- data collection
- question
- hypotheses
The variables in the study phase allows for ____-______ the variables, the _____ or ______, and specify ____ ____.
- cross-referencing
- questions
- hypotheses
- survey items
Steps for analyzing the data in data analysis and interpretation phase:
- report response rate
- determine response bias: the effect of nonresponses on survey estimates
- discuss plan to provide descriptive analysis
- check instrument’s scales
- statistics and statistical computer program for inferential statistical analyses
- present and interpret results
The the data analysis and interpretation phase, report how the results answered the _____ ____ or _____.
- research question
- hypotheses
The the data analysis and interpretation phase, discuss the ______ for the results for practice or _____ research on the topic. Draw _____ and ____ from results.
- implications
- future
- inferences
- conclusions
2 questions asked when looking at error in survey research:
- how closely the responding sample is representative of the population
- how well the questionnaire answers measure the characteristics to be described
2 errors related to how closely the responding sample is representative of the population:
- sampling error
- frame (coverage) error
sampling error:
occurs when a sample drawn from the population is not representative of the entire population
Frame (coverage) error:
occurs when the list from which the sample is drawn fails to contain all members of the population
2 errors related to how well the questionnaire answers measure the characteristics to be described:
- response error
- nonresponse error
Response error:
- occurs when the respondent is unable or unwilling to respond to all items in a survey as a result of the communication process
- do not answer truthfully to items
Nonresponse error:
extent to which individuals included in the sample fail to provide usable responses (systematically different from those who respond to the survey)
Falling response rates:
less effective surveys = % of surveys that are returned compared to the # sent out
potential reasons for falling response rates:
- changing cultural norms of cooperation
- overexposure to academic and marketing surveys
- increasing concerns regarding confidentiality
Benefits of internet surveys:
- potentially allow to reach more of the population
- 24/7 data collection
- ease of delivery
- easy storage
- provide less threatening approach for sensitive data
Costs of internet surveys:
- sources of error (frame, sampling, measurement)
- ethical issues
- level of expertise required
