Chapter 5 Flashcards
Survey Research
- Describe thoughts, opinions, feelings
- Measure naturally occurring variables
- Allow predictions based on correlations
- Questionnaires
- Predetermined set of questions
- Sample represents a population
- Examine survey procedures and analyses for sources of bias
Sampling in Survey Research
- Use sample to represent the larger population
- “Representative”: similar to
- Requires careful selection of a sample
- Goal: Generalize survey findings from representative sample to the population
Biased Samples
- Characteristics of the sample differ systematically from those of the population.
- Sample over-represents or under-represents segment(s) of a population
Selection Bias
Researcher’s procedures for selecting sample cause bias
Response-rate bias
Individuals selected for the sample do not complete the survey
Populations
Set of all cases of interest
Sampling frame
We need to develop a specific list of members of the population in order to select a subset of that population
Sample
The subset of the population actually drawn from the sampling frame
*Representative of the population to the extent that it exhibits the same distribution of characteristics as the population
Representativeness
The ability to generalize from a sample to the population
Probability sampling
By selecting students randomly from the registrar’s list of students, each person (element) on the list has an equal chance of being included in the sample
- Far superior to nonprobability sampling in ensuring that selected samples represent the population
- All members of population have a specified chance of being selected for the survey
Nonprobability sampling
Does not guarantee that every element in the population has an equal chance of being included in the sample
*Sample likely not representative of population
Simple random sample
Random selection, random-digit dialing
- Basic technique of probability sampling
- Every element has an equal chance of being included in the sample
1. Define your population of interest
2. Determine your sampling frame
3. When a convenient list of manageable size is available, a random sample can be taken to achieve a representative sample.
Stratified random sample
The population is divided into subpopulations called strata (stratum) and random samples are drawn from each of these strate
- Two general ways to determine how many elements should be drawn from each stratum
- Draw equal-sized samples from each stratum
- Draw elements for the sample on proportional basis
- Only the stratified sample on a proportional basis would be representative.
Convenience sampling
Individuals are available and willing to respond to the survey
- Example: magazine surveys, call-in radio surveys
- Will result in a biased sample unless you have strong evidence confirming the representativeness of the sample
Four methods for obtaining survey data
- Mail surveys
- Personal interviews
- Telephone interviews
- Internet surveys
Ways to increase response rate
- Questionnaire has a “personal touch”
- Use name, not “Dear student”
- Responding requires minimal effort
- Topic of survey is interesting to respondents
- Respondents identify with organization or sponsor of survey
“Research design”
- A plan for conducting a research project
* Choose method best suited for answering a particular question
Three types of survey research designs
- Cross-sectional design
- Successive independent samples design
- Longitudinal design
Mail Surveys
Distribute self-administered questionnaires that respondents fill out on their own
*Advantage: they usually can be completed relatively quickly
-Avoid the problems due to interviewer bias
-Best for dealing with highly personal or embarrassing topics, especially anonymity of respondents is preserved.
Disadvantages: The cost of copying and mailing
-Self-explanatory
-The researcher has little control over the order in which the respondent answers the questions
-The order of questions may affect how respondents answer certain questions
-Low response rate-> response rate bias
Response rate bias
threatens the representativeness of a sample
Personal Interviews
- Allows greater flexibility in asking questions
- Ensure that all respondents complete the questions in the same order
- Higher response rate
- Allow researchers to gain more control over how the survey is administered
- Disadvantages: Costly
- Interviewer bias
Interviewer Bias
- Occurs when the interviewer records only selected portions of the respondent’ answers or tries to adjust the wording of a question to “fit” the respondent.
- Protection against interviewer bias: employ highly motivated, well-paid interviewers who are trained to follow question wording exactly, to record responses accurately, and to use follow-up questions judiciously.
- Interviewers should also be given a detailed list of instructions about how difficult or confusing situations are to be handled.
- Interviewers should be closely supervised by the director of the survey project.
Telephone Interviewers
- Are used frequently for brief surveys
- Random-digit dialing techniques
- Provides better access to dangerous neighborhoods, locked buildings, and respondents available only during evening hours.
- Can be completed more quickly
- Interviewers can be better supervised
- Limit to how long respondents are willing to stay on the phone
- Respond differently when talking to a “faceless voice”
Internet Surveys
- Participants complete a questionnaire online and click on a “submit” button to have their responses recorded.
- Permit manipulation of variables and the random assignment of participants to experimental conditions
- Advantages: efficiency, cost, access groups
- Disadvantages: Potential for sample biases
- No way to generate a random sample of Internet users
- Response rate bias and selection rate bias
Cross-Sectional Survey
*Select sample from one or more populations at one time
*Survey responses are used to
-Describe population (descriptive statistics)
-Make predictions for the population (correlations)
at that one point in time
*Compare populations
*Cannot assess change over time
Successive Independent Samples Design
- A series of cross-sectional surveys over time
- A different sample from the population completes the survey each time.
- Each sample is selected from the same population.
- Responses from each sample are used to describe changes in the population over time.
- Problem: noncomparable successive samples
Longitudinal Survey Design
- Same sample of individuals completes the survey at different points in time
- Assess how individuals change over time
- Responses from the sample are generalized to describe changes over time in the population.
- Problems: attrition and reactivity
Questionnaires
- Most frequently used to collect survey data
- Measure different types of variables
- Demographic variables using checklists
- Preferences, opinions, and attitudes
- -Self-report scales
- -Rating scales (assume interval level of measurement)
- All measures must be reliable and valid.
Reliability
Consistency of measurement
Test-retest reliability
- Administer measure two times to same sample
* High correlation between the two sets of scores indicates good reliability (r > .80)
How to improve reliability?
- More items
- Greater variability among individuals on the factor being measured
- Testing situation free of distractions
- Clear instructions
- A measure can be reliable but not valid.
Validity
- the truthfulness of a measure.
* Assesses what it is intended to measure
Construct validity
*Instrument measures the theoretical construct it was designed to measure.
Convergent Validity
*Extent to which two measures of the same construct are correlated (go together)
Discriminant Validity
*Extent to which two measures of different constructs are not correlated (do not go together)
Constructing a Questionnaire
- Best choice for selecting a measure
- Use measure already shown to be reliable and valid in previous research.
- If no suitable measure is found, create a questionnaire or measure.
- Creating a reliable and valid questionnaire is not easy.
Important first steps when constructing a questionnaire
- Decide what information should be sought.
- Decide how to administer the questionnaire.
- Write a first draft of the questionnaire.
- Reexamine and revise questionnaire based upon expert advice.
- Pretest the questionnaire.
- Review results and edit the questionnaire.
Guidelines for Writing Survey Questions
- Choose how participants will respond
- Free-response or closed questions
- Use simple, familiar vocabulary
- Write clear, specific questions
- Avoid double-barreled questions
- Place conditional phrases at the beginning of sentences
- Avoid leading questions and loaded questions
- Avoid response bias
Ordering of questions
- Self-administered questionnaires
- Place most interesting questions first
- Personal and telephone interviews
- Demographic questions first
- Use funnel questions and filter questions as needed.
Thinking Critically About Survey Research
- Correspondence Between Reported and Actual Behavior
- Survey responses may not be truthful.
- Reactivity
- Social desirability
- Accept people’s responses as truthful unless there’s reason to suspect otherwise.
- Use a multimethod approach to answering research questions.
Mediators
Variables used to explain a correlation between two variables
Moderators
Variables that affect direction or strength of correlation between two variables
- may affect the direction and strength of these relationships.
- Sex of the child
- Population density (e.g., rural, urban)
- Personality features of children (e.g., resilience)
Demographic Variables
Describe the characteristics of the people who are surveyed
Self-report scales
Measure people’s judgments about items presented on the scale or to determine differences among people on some dimension presented on the scale
