15.10 Data collection Flashcards
1.Survey Research?
Variables are measured by asking respondents questions using standardized questionnaire or interview procedures.
Approach to asking questions is usually structured.
Answers are numerically coded (even nominal variables) for analytic purposes.
2.Survey Data Collection Modes?
Mail Self-administered in group settings Internet Telephone Personal (face-to-face)
Some Issues to 3.Consider in Choosing Data Collection Methods?
Cost Geographic location of respondents Speed of turnaround Length and complexity of questionnaire Sensitivity of topic
4.Total Survey Error?
Coverage error Sampling error Nonresponse error Item nonresponse errror Measurement error
5.Method of Reporting?
Paper and pencil
Technology assisted
Computer assisted
-CASI (computer-assisted self-interviewing)
-CATI (computer-assisted telephone interviewing)
-CAPI (computer-assisted personal interviewing)
- What Is an Acceptable Response Rate?
Response rate = # surveys completed/ total eligible sample
>=85% Excellent 70-85% Very good 60-70% Acceptable 50-60% Barely acceptable <50% Not acceptable
- Mail Surveys and Self-administrated Surveys?Advantages
Lower cost than telephone or personal Geographic dispersion okay Privacy for respondents Convenience for respondents No interviewer bias This method is reliable when closed questions are used, when the order in which questions are answered is unimportant, and when the questions and format are simple and straightforward.
- Mail Surveys
- Disadvantages?
Potential for coverage error Potential for nonresponse error Potential for item nonresponse error Length, complexity limits Open-ended questions problematic Selection factors (e.g., literacy) No potential of interviewer clarification/assessment
- Procedures/Practices for Ensuring Acceptable Response Rate?
Cover letter to respondent Postage on return envelope Guarantee of confidentiality or anonymity Reminders Incentives Length of questionnaire Clarity of instructions Aesthetics
- Example: Mail Survey?
School-based Substance Use Prevention Programs Study (SSUPPS)
Sample design: Stratified random sample of public and private schools
Respondents: Middle school teachers
Data collection: Mailed survey
- Example Data Collection Schedule for SSUPPS?
3/03/07 Lead letter sent to teachers 3/10/07 1st questionnaire sent by mail 3/17/07 Reminder postcard 3/31/07 2nd questionnaire sent by mail 4/14/07 Telephone prompt 4/28/07 3rd questionnaire sent by mail 5/12/07 Postcard w/calling hours
- Telephone Surveys
- Advantages?
Lower cost than personal interviews
Quality control is much easier than personal interviews
Speed
Can utilize special question formats in which certain questions are skipped, while such a format may be confusing for respondents completing a questionnaire
Cost is about 45 to 64 percent of personal interviews
In terms of sampling quality, the telephone survey mode is between personal interviews and mailed questionnaire
- Telephone Surveys
- Disadvantages?
Potential for coverage error
Potential for nonresponse error (but may be less than for mail surveys)
Reluctance of respondents to answer sensitive questions
Length/complexity limits
Open-ended questions problematic
Hard to establish trust and rapport with respondents than in personal interviews
- Personal Interviews (Face to Face)
- Advantages?
Higher response rate than mail or telephone (range from 85 percent in rural counties to 67 percent for large central cities in US)
Less potential for item nonresponse than mail or telephone
Appropriate for longer/more complex surveys (90 min for completing of some 400 questions)
May be appropriate for sensitive topics
Can use visual aids
Interviewer can clarify questions
Opportunity of unobtrusive observation
Can utilize special question formats in which certain questions are skipped
- Personal Interviews
- Disadvantages?
Cost (more than two times of cost in mailed questionnaire)
Quality control less easy than in telephone surveys (interviewers, data coders, and the researcher are geographically dispersed)
Difficulty locating respondents at home
Potential for interviewer bias
- Interviewer Bias?
Anything that creates a systematic difference between responses obtained by different interviewers
Respondent’s perceptions of the interviewer and his/her reaction to that
Female interviewers may be perceived as less threatening; older as more judgmental
Interviewer’s perceptions of the respondent and his/her reaction to that
17.Characteristics of a good interview?
An appropriate atmosphere-friendly
Respondent is at ease
Clarification is obtained for confusing answers
Interviewer gives only neutral responses to the respondent’s answers
The response is accurately recorded
18.Specific skills required for interviewers?
Be able to ask questions at the correct pace and in a conversational tone
Know the questions and response categories well enough to keep the interviewing flowing smoothly
Know when there are probes that can be used and how to use them
Be able to think as an interviewer and put aside other roles
Be able to maintain a positive attitude so respondent feels interview is important
Be able to keep control over the interview process
19.Additional requirements for interviewers?
Mobility – for personal interviews
Flexibility over schedule
Neat, pleasant, professional appearance
Not too timid, not too aggressive
- Internet Surveys?
World Wide Web
- Internet Administered Surveys?
Emerging technology
Use in measurement construction/validation
Questionnaire sent directly to e-mail addresses or volunteers sought (e.g., through news groups)
Many of same advantages as mailed surveys and faster turnaround
Many of same disadvantages of mailed surveys, especially potential for coverage error
Item nonresponse error may be lower than in mail surveys
- Internet Surveys
- Advantages?
Cost saving (printing, postage, data entry) Quick access and response time Easy to cross international boundaries Possibility for large sample sizes Interactive potential with Web surveys
- Internet Survey
- Disadvantages?
Requires a computer Computers differ in hardware/software capabilities Potential for coverage error Difficulties with sampling Unknown generalizability of results
- Procedures/Practices for Ensuring Acceptable Response Rate?
Follow same strategies as for mailed surveys
Use multiple contacts
Personalize
Brevity and simplicity in e-mail surveys
Don’t use most advanced technology in Web surveys (some computers may have lower version of web or people may not be able to understand those fancy technology)
- Quality Control of the Whole Data Collection Process?
Select the data collection method that is most appropriate to the particular need (i.e., purpose) and target population for the study
Develop and pretest the instrument and data collection procedures
Select data collectors with needed skills and provide training
Develop detailed data collection protocols
Monitor for quality control
- Preparation of Data Collection Materials?
Consent materials Questionnaire Contact sheets and tracking forms Data collection protocol instructions Other forms (Administrative forms) Supplies (Pen, highlight et al.)
- Data Collection Protocols?
Detailed instructions for interviewers/data collectors to follow
What to say in instructing people to complete the questionnaire or other data collection procedures
What questions are likely to occur and how to answer
How to handle problem
Instructions for obtaining and contacting participants
Scheduling information (e.g., sequencing of events, when to contact)
- Training
- Include multiple components?
Study goals, significance, sponsor, who’s conducting
Interviewing techniques
Data collection procedures
Consent procedures, confidentiality
Familiarity with the questionnaire and special instruction
Demonstration and mock interviews
Administrative issues
