Designing Research Instruments Flashcards
1
Q
CATI
A
Computer-Assisted Telephone Interviewing (CATI)
“A data‐collection technique in which
a telephone‐survey questionnaire is stored in a computer, permitting the interviewer to read the questions from the monitor and enter the answers on the computer keyboard. See Chapter 8.”
- Some application that shows interviewers the questions and they’ll record answers
- Developed in the late 70s
- The key technique developed in the 40s
- Key advantages are
- Reduced traveling time
- Immediate responses
- Pre-programmed questions in an application
- Hard work to be an operator
- Refused often; people not polite
- Easy to supervise; people in one room interviewing (call centers)
- Supervisors might hear the interviews
- Checking if operators are asking questions correctly
- Might be able to see screen that the operator is looking at
- Checking that operators are accurately recording answers
- Often using RDD (Random Digit Dialing)
- Automatic by applications
2
Q
F2F, PAPI, CAPI
A
F2F = Face to Face; always quantitative
PAPI = Paper And Pencil Interviewing
CAPI = Computer-Aided Personal Interviewing
- Core, universal technique of data collection
- Sometimes people don’t see the questionnaires; sometimes it doesn’t matter
- Traditional PAPI interviews now mostly replaced by CAPI
- The interviewer must travel to respondents
- Interviewers in the home; natural environment
- CAPI advantage - you don’t have to rewrite info from paper questionnaires
- Tablets, etc. equipped with GPS and timers to ensure basic details properly recorded
- May also use audio recordings during sessions (with permission)
Paper, you have to rely on verbal responses
- including where it took place, how long was the interview, etc.
- people sometimes lie
- 40-50% refusal rate
3
Q
SAQ
A
Self-Administered Questionnaire (SAQ)
Deliver questionnaires to respondants
- By mail, fax, etc.
- May walk around and put in mailboxes
- Respondents answer and send back
- Nobody who can help respondent with questions
- Must be very clear so people understand in a proper way
- Must be nice-looking, easy to fill-in, must include instructions
- Must figure out how you’ll get those questionnaires back
- Mail, personal collection, drop-off box, etc.
- Must be easy and comfortable for respondents or they won’t participate
- Used less often than in the past; replaced with CAWI
4
Q
CAWI
A
Computer-Aided Web Interviewing (CAWI)
- Generally replaced SAQ
- Some don’t have email, computers, smartphones, etc.
- Contact information is an issue
- Must be careful with privacy and personal data protection
- Must follow regulations in the country where you’re interviewing
- Allow different types of questions, more complex branching, etc.
- Receive responses in real-time
5
Q
Classification by structure
A
✓✓ = more frequent ✓ = less frequent
Structured:
- Descriptive ✓✓
- Explanatory ✓
Semi-structured:
- Explanatory ✓✓
- Exploratory ✓
In-depth:
* Exploratory ✓✓
6
Q
Modes of research methodology differ in… (5 things)
A
- Degree of interviewer involvement
- Degree of interaction with the respondent
- Degree of privacy
- Channels of communication
- Use of technology
7
Q
Degree of Interviewer involvement
A
- If you know you’ll have to clarify some of the things you’re asking, you might want to have a more interactive interviewer
- If interviewers aren’t involved, you can’t be sure how good your responses are or how well respondents understood questions
- Interviewers might check certain issues or criteria when interviewing
- Ex: place where someone is living; state of a person’s home
- You might research sensitive topics and could be a problem for respondents to talk about these things
- May be willing to provide answers in a questionnaire, but not willing to talk about it in person
- Not pleasant for people to even pronounce certain words
- Feels safer to answer in written form
- Social desirability effect
8
Q
Social desirability bias
A
“In social science research, social desirability bias is a type of response bias that is the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. It can take the form of over-reporting “good behavior” or under-reporting “bad,” or undesirable behavior.”
9
Q
Degree of interaction with the respondent
A
- If you want someone to respond to a jingle, you would have a hard time programming an application to record an accurate response
- If asking a question with 7 options, you would have to show them the 7 options because they can’t properly remember them alone
10
Q
Channels of communication
A
- Can you deliver the type of information that you need
- Can you collect responses back?
- If face-to-face interviews are more effective, people need to see each other and communicate
- Some remote channels are ok, maybe better
- May need to reach someone in a different country
11
Q
Use of technology
A
- Still maybe people who may visit with a tablet or compute, and could make people more nervous, surprised, scared
- Openness lowered because of those tools (ex: older generations)
- May cause specific types of biases
12
Q
F2F & CAPI
Advantages
A
- Generally yields highest cooperation and lowest refusal rates
- If a low number of people in some field, these rates very important
- Allows for longer, more complex interviews
- Takes advantage of interviewer presence
- Observe facial expression,
- Often some questions at the end for the interviewer to answer about fear, attitude, tiredness, concentration, other people in the room, etc.
- Literacy levels are not a major concern
- You can interact with the respondent (ex: ask follow-up questions, clarify points, and probe)
- Respondents can’t skip questions or respond with “idk” or “idc”
- The respondent may provide new contacts (ex: in case of in snowball)
- Greater comfort for the respondent (particularly in home settings)
- Doesn’t have to read, travel, etc
- No obvious selection biases
13
Q
F2F & CAPI
Disadvantages
A
- Most costly mode (unless at sites); time and effort required to set-up and conduct interviews may not match the “pay-off” in terms of useful information
- Countries differ in this - travel cost
- Buying tablets
- Longer data collection period; they tend to be time-consuming
- Interview itself, writing transcript, process data = lots of time
- Interviewer concerns (Bias)
- People make mistakes (sometimes unintentionally, sometimes on purpose if uncomfortable)
- Interviews can be stressful for both parties
- Typically evening at home; unexpected; intrusion
- If people accept, maybe worried about performance; what interview will be like, etc.
- Could have an uncomfortable home situation for the interviewer
- Quality and training for interviews plays a major role in success
- Unexperienced or poorly trained interviewers could negatively impact findings
- Initial contact is frequently difficult; contacting the specific person in the house
- The way you introduce yourself really matters
- Need to achieve highest possible participation rate; depends on training
- Creating rapport
- Confidentiality may be harder to assure
- Hard to persuade about confidentiality
- Obviously not anonymous if you’re coming to their house; know their name and address
- Can only declare that the contact info will remain confidential and won’t be misused
- Difficulty with interviewer supervision
- Can’t see when interview starts, ends, etc.
14
Q
CATI Advantages
A
- Less expensive than personal interviews
- Shorter data collection period than personal interviews
- Better control and supervision of interviewers (vs. personal)
- Interviewers contained; can see them; can provide immediate feedback
- Literacy levels are not a major concern
- CATI systems allow for a very high rate of accuracy in transcribing answers and preparing data
- Open-ended questions - can’t write fast enough on the keyboard to record response; written with tons of mistakes or answers aren’t complete
- Most systems enable voice recording
- Can easily generate a national or statewide sample
- Difficult to run probability samples with F2F interviews
- Can use RDD - easier for probability
15
Q
CATI Disadvantages
A
- Biased against households without telephone, unlisted numbers
- Issue of calling cell phones
- People could be in many places when you call; might be disturbing to people
- Questionnaire constraints
- If you have many different options, it’s hard for them to remember all options; if a question requires a showcard (“look at this ad”) you can’t use CATI
- Difficult for sensitive questions or complex topics
- High refusal rates may lead to a biased sample
- Very high refusal rates
- Higher educated are often busier people and will be excluded
- Easier to refuse person over phone than in person
- 10-15% response tate
- Even when people do not refuse, mistrust is common
