test 1, survey research Flashcards
Identify the steps in the survey research process.
1. Data collection (1) Sampling (2) Questionnaire design or survey instrument design) (3) Conducting the survey Survey modes: (a) mail (b) telephone (c) in-person (d) Internet 2. Data entry and data processing 3. Data analysis 4. Survey report (description of the survey process, major findings, conclusions, etc.; diagrams, charts, tables, etc.will be used)
Define and find examples of sample.
part” or subset of population
What are the major similarities between census and survey?
Both census and survey use the same basic techniques and methods for data collection, data entry and processing, data analysis, and findings reporting.
Discuss the widespread use of sampling and survey by other academic disciplines and professions.
costco, schools, marketing
When can errors occur in the survey process?
- Using “part” (sample) to estimate or project
- Using a “whole”( population )to estimate or project
- Respondent answers are not accurate measures of the characteristics of the respondents.
- Sample does not represent population.
Discuss various types of biases in surveys.
- Incomplete sample frame: Sample frame does not include all the people to be sampled.
- Sampling is not random.
- Incomplete answers, non-response, and low completion rate.
Differentiate between random error and systematic error in sampling Find real life examples for each.
sampling is concerned with the selection of a subset of individuals from within a statistical population to estimate characteristics of the whole population
systematic error is called a bias ex. is an internet survey
Define sampling.
sampling is concerned with the selection of a subset of individuals from within a statistical population to estimate characteristics of the whole population
What is the relationship between a sample and the population
a sample is a subset of a population
Identify and find examples of the major sampling methods. And discuss their appropriate uses.
1 stage sampling
a. Simple random sampling:
b. . Systematic sampling
c. Stratified sampling
2. multistage sampling
a. Area probability sampling
b. Random-digit dialing (RDD)
Why is stratified sampling needed
When the distribution of the target population on a particular indicator (e.g., gender, grade level) is known, stratified sampling can give all sub-groups equal representation in the sample through unequal weighting of different strata
What does the chance of selection refer to?
Each individual in the population of interest has an equal likelihood of selection.
What are the factors affecting sample quality?
Feasibility
Cost
Precision (how large a sampling error can be tolerated)
What are the characteristics of a good sample frame?
Comprehensiveness (must include all the people to be sampled)
A person’s probability of selection (to be selected) can be calculated.
Efficiency (the people to be sampled can be found easily and cost-effectively)
Identify the major causes of non-response.
Interviewee cannot be reached.
Interviewee refuses to answer questions.
Interviewee cannot provide data due to difficulties (e.g., illness, language problems, etc.)
sensitive questions
How samples are drawn in Systematic sampling
- Decide sample size.
- Divide the total population into equal “blocks” by the sample size.
- Select a random number within the boundaries of one block and use it SYSTEMATICALLY throughout for ALL BLOCKS.
Under what circumstances do people are most likely to respond to a survey?
interest and relevance
How to calculate response rate
people who complete the survey / total number of people surveyed.
Generally speaking, which groups (in terms of age, education, immigration status, etc.) are more likely to respond in surveys?
older educated natives
Generally speaking, which area (urban vs. suburban) tends to have a higher response rate?
suburban
How to reduce non-response in interviewer-administered surveys?
make appointments, know your timing
Discussthe nature and effects of non-response
Non-response error is hard to measure. There is still no clear evidence on when and how much non-response will or will not affect survey estimates.
The effect of non-response on one variable can be different than for others in the same survey.
No one knows how biased non-response is, but you have to assume it is biased.
Discuss the merits and weaknesses of internet surveys.
most biased
reach only part of the population
people have multiple email accounts
cheap can get to a lot of people
How to reduce non-response in mail surveys?
(1) Questionnaire must be easy to see, easy to read and easy to answer.
(2) Small cash payment improves response rate significantly.
(3) Repeated contacts and telephone follow-ups are needed.
What are the good practices to boost response rate?
(1) Combine 2-3 modes or methods to produce a higher response rate. For example: use mail or email for first phase of data collection, followed by telephone interviews with non-respondents.
(2) Data collected from follow-ups using a different mode need to analyze to make sure they are comparable with previously collected data.
Identify self-administered, surveys.
mail, internet, and computer-assisted surveys.
Identify interviewer-administered, surveys.
telephone and personal interview surveys.
Identify group-administered surveys.
college classroom
Identify the major answer errors respondents are likely to make.
(1) Misunderstanding of survey questions
(2) No knowledge or not sure
(3) No standard measure for subjective feelings
(4) Overstatement, understatement or pure lying
What does reliability refer to?
providing CONSISTENT measures
Define and find examples of validity.
measure what it is supposed to measure; using the right measure
What are the ways to increase reliability?
- Ask the same set of questions for each respondent.
- Wording of the questions must be universally understood
- AVOID complicated and unfamiliar words and phrases, poorly defined terms, and incomplete sentences.
- Use optional wording as needed.
- Avoid asking multiple questions in a single sentence.
- In case translation is needed, use special wording for special subgroups to avoid misunderstanding or loss of meaning.
- Whenever possible, use closed questions and provide respondents with a list of acceptable or expected answers.
How to increase validity?
- Validity: Right measure for right subject.
- Validity is somewhat different for subjective and objective measures.
- Validity of opinions and feelings is hard to assess. It can only be assessed in relation to other answers a respondent gives.
- Whenever possible, use closed questions.
- Provide definitions to complex terms to reduce answer error.
- change the question to ask information that is less detailed and easier to recall or estimate.
- Tuse self-administered survey methods and assure confidentiality and anonymity.
- Ask several questions with different question forms that measure the same subjective matters.
What are open-ended questions and their appropriate uses?
This type of question allows participants to respond in any way they choose
used in exploratory research.
What are the major types of closed-ended questions and their appropriate uses?
Multiple-choice-Use a multiple-choice question when you want your respondents to choose the best possible answer among all options presented.
Nominal (categorical or dichotomous)-Use a categorical question when the possible answers are categories, and the respondent must belong to one category.
Ordinal-Sometimes you may want your respondents to rank order their responses. A ranking indicates the importance assigned by a participant to an attitudinal object.
Likert scale- one standardized form of ordinal question format. Use a Likert-scale question when you are trying to determine respondents’ attitudes or feelings about something.
Semantic differential (one form of ordinal question format)-opinions very much like to very much hate
Guttman scale (one form of Semantic-Here, the respondent checks each item with which they agree. The items themselves are constructed so that they are cumulative – if you agree to one, you probably agree to all of the ones above it in the list.
Numerical-
Ordinal
provide primarily quantitative data, and are frequently used in confirmatory research.
What are contingency questions and their appropriate uses?
Sometimes you have to ask the respondent one question in order to determine if they are qualified or experienced enough to answer a subsequent one
What are matrix questions and their appropriate uses?
The matrix question is an efficient format for presenting several items sharing the same response categories.
What are the major differences between census and survey? With the subject matter
Census Survey
Subject: IndividualsIndividuals, organizations, or other
What are the major differences between census and survey? With the data collected
Census Survey
Data collected: Facts only Opinions, facts, or both
What are the major differences between census and survey? With the cycle(how often)
Census Survey
Cycle: Every 10 years Anytime
What are the major differences between census and survey? With the sponsors
Census Survey
Sponsor: Census Bureau Any organization or individual
What are the major differences between census and survey? With the expense
Census Survey Expense: Expensive Inexpensive
What are the major differences between census and survey? With the scale
Census Survey
Scale: Large Small
Identify the major answer errors respondents are likely to make.
(1) Misunderstanding of survey questions
(2) No knowledge or not sure
(3) No standard measure for subjective feelings
(4) Overstatement, understatement or pure lying
How samples are drawn in Stratified sampling
Before sampling, the population is divided into characteristics of importance for the research. For example, by gender, social class, education level, religion, etc. Then the population is randomly sampled within each category or stratum. If 38% of the population is college-educated, then 38% of the sample is randomly selected from the college-educated population.
How samples are drawn in Simple random selection
In a simple random sample (SRS) of a given size, all such subsets of the frame are given an equal probability. Each element of the frame thus has an equal probability of selection
How samples are drawn in multistage selection
Stage 1: Select a small number of “clusters” (geographic areas, organizations, etc.)–NOT INDIVIDUALS–from all the “clusters” as sample. At this stage: Population—all “clusters” Sample–some “clusters”
Stage 2: Select individuals from the selected “clusters” At his stage: Population–all individuals in the selected “clusters” Sample–some individuals selected from these clusters
How to calculate non-response rate?
total number of people surveyed/people who complete the survey
In group-administered surveys, respondents answer questions (record answers) just like in _______.
self report
Discuss the suitable uses for each survey mode.
mail
telephone
in-person
Internet
Discuss issues about questions format,
open-ended-coding is impossible, only use when exploring new issues, and a small population
close ended-yes no questions
What is the difference between error and bias
error is random it can go both ways. bias is consistent can go only one way
What kind of errors are there
- human
- equipment
- answer either lack of knowledge or misunderstanding
- unrepresented sample
what is a sample frame
as close to the population as you can get
Discuss issues about , costs,
costs from high to low: personal interview survey telephone survey group-administered survey mail survey Internet and computer-assisted surveys.
Discuss issues about response time,
the order of response rate–from high to low– looks like the following: group-administered survey personal interview survey telephone survey Internet and computer-assisted surveys mail survey
Mail and personal interview surveys are not suitable for fast response.
Discuss issues about equipment requirements,
Internet and computer-assisted surveys have higher requirements on facilities and are not suitable for sparsely populated areas and small cities where high-speed Internet service and computer labs are not readily available.
Discuss issues about content
Closed-end questions are preferable, especially for mail survey.
Break down complex questions into a set of simpler ones.
Sensitive questions are more suitable for mail, Internet and computer-assisted surveys.
