PROCTOR EXAM 1 - Quantitative Data Collection Methods Flashcards
Telephone
Interviews which are conducted person-to-person, via the telephone.
Interviews which are conducted person-to-person, via the telephone.
Achieving a good response rate is a combination of two main factors, interviewer skill and good fieldwork practice.
Computer Assisted Telephone Interview CATI - Telephone research
Computer Assisted Telephone Interview CATI - Telephone research
Dialers (automatic dialing systems machines) are often employed by telephone centers to dial the telephone numbers and route the answered call to an available interviewer
Dialers (automatic dialing systems machines) are often employed by telephone centers to dial the telephone numbers and route the answered call to an available interviewer
TELEPHONE
PROS
*Sample control: efficiency - telephone ownership (landline, cell or VOIP) is almost universal, there are no geographical restrictions.
Sample control: sample frames - RDD (Random Digit Dialing) including mobile phone numbers makes frame errors potentially close to zero.
Field force control - interviewers can be supervised and monitored in real time.
Speed - large number of interviewers employed on a project can complete reasonably large samples relatively quickly.
Sensitive information - interview is reasonably anonymous and perception is that nothing is being committed to over the phone.
Low incidence rate - large numbers of screening calls can be made with reasonable efficiently and at relatively low cost.
TELEPHONE
PROS
*Sample control: efficiency - telephone ownership (landline, cell or VOIP) is almost universal, there are no geographical restrictions.
Sample control: sample frames - RDD (Random Digit Dialing) including mobile phone numbers makes frame errors potentially close to zero.
Field force control - interviewers can be supervised and monitored in real time.
Speed - large number of interviewers employed on a project can complete reasonably large samples relatively quickly.
Sensitive information - interview is reasonably anonymous and perception is that nothing is being committed to over the phone.
Low incidence rate - large numbers of screening calls can be made with reasonable efficiently and at relatively low cost.
TELEPHONE
CON
Diversity of questions/flexibility - restricted by participant’s ability to remember the answer choices or do the task. Ranking exercises and ‘single choice from many’ are impossible.
Physical stimuli - cannot be used except in a multi-stage project (recruit – mail out – recall).
Quantity of data - participant can break off the survey relatively easily; telephone interviews require concentration and are tiring, therefore data quality will deteriorate with long interviews due to satisficing.
Response rate - is becoming a problem due to sugging and frugging by non-research organizations and increased screening of calls via caller ID.
Participant control - the interviewer dictates the pace and the timing (at least of the initial contact).
TELEPHONE
CON
Diversity of questions/flexibility - restricted by participant’s ability to remember the answer choices or do the task. Ranking exercises and ‘single choice from many’ are impossible.
Physical stimuli - cannot be used except in a multi-stage project (recruit – mail out – recall).
Quantity of data - participant can break off the survey relatively easily; telephone interviews require concentration and are tiring, therefore data quality will deteriorate with long interviews due to satisficing.
Response rate - is becoming a problem due to sugging and frugging by non-research organizations and increased screening of calls via caller ID.
Participant control - the interviewer dictates the pace and the timing (at least of the initial contact).
TELEPHONE (GLOBAL) Mobile subscribers, in many developing countries, outnumber fixed line subscribers.
TELEPHONE (GLOBAL) Mobile subscribers, in many developing countries, outnumber fixed line subscribers.
Telephone is well suited to B2B interviewing, since it is a natural medium for most business target audiences. Often, however, the biggest challenge in B2B is finding the correct person to interview within the organization. This is more easily done by telephone than by other methods.
Telephone is well suited to B2B interviewing, since it is a natural medium for most business target audiences. Often, however, the biggest challenge in B2B is finding the correct person to interview within the organization. This is more easily done by telephone than by other methods.
Ethical Considerations
Abiding by any local ethical guidelines, interviewers need to be prepared and able to tell participants where their telephone number has been obtained, the nature and purpose of the research, and in some cases be able to point the participant to a third party who will be able to confirm the legitimacy of the research company.
Ethical Considerations
Abiding by any local ethical guidelines, interviewers need to be prepared and able to tell participants where their telephone number has been obtained, the nature and purpose of the research, and in some cases be able to point the participant to a third party who will be able to confirm the legitimacy of the research company.
Response Rate Considerations
Achieving a good response rate is a combination of two main factors, interviewer skill and good fieldwork practice.
Response Rate Considerations
Achieving a good response rate is a combination of two main factors, interviewer skill and good fieldwork practice.
Quantitative Observation
simple counting of the
prevalence of an attribute.
Quantitative Observation
simple counting of the
prevalence of an attribute.
Quantitative Observation
Mystery shopping , where the observer (‘ shopper ‘) poses as a customer to assess quality of service and standards compliance, is a form of Observational research.
Quantitative Observation
Mystery shopping , where the observer (‘ shopper ‘) poses as a customer to assess quality of service and standards compliance, is a form of Observational research.
Observation research , like
passive measurement, gets
away from the problem of
awed recall by counting actual behavior, occurring in its natural setting.
Observation research , like
passive measurement, gets
away from the problem of
awed recall by counting actual behavior, occurring in its natural setting.
Observation is not constrained by sampling, survey design or analysis issues. The observer simply sets up and counts.
Observation is not constrained by sampling, survey design or analysis issues. The observer simply sets up and counts.
Observers may also unwittingly impart an ‘observer bias’ if they are asked to make judgments about those they observe.
Observers may also unwittingly impart an ‘observer bias’ if they are asked to make judgments about those they observe.
B2B CONSIDERSATION OBSERVATIONAL
Businesses may also be less than enthusiastic about opening their procedures up to scrutiny from what may very well be a competitor.
B2B CONSIDERSATION OBSERVATIONAL
Businesses may also be less than enthusiastic about opening their procedures up to scrutiny from what may very well be a competitor.
Question Diversity and Flexibility: More complex question types can be undertaken when there is an
interviewer present to help explain the task or when the questionnaire is visible to the participant.
Questionnaires may become more complex if intricate routing and survey logic is required.
Question Diversity and Flexibility: More complex question types can be undertaken when there is an
interviewer present to help explain the task or when the questionnaire is visible to the participant.
Questionnaires may become more complex if intricate routing and survey logic is required.
Physical Stimuli: The need for this can range from the most basic (a visual representation), to touch and possibly also to smell and taste.
Physical Stimuli: The need for this can range from the most basic (a visual representation), to touch and possibly also to smell and taste.
Sample Control: Defined as the ability of the method to reach the participants specified by the sample
definition. It can be broken down further into three useful components:
Theoretical Coverage: Is it possible, in theory, to reach all potential participants?
Sample Frames: To what extend do the lists you have available to you reduce the actual coverage?
Efficiency: Can the above two factors be mitigated by greater efficiency—the ease with which
qualifying participants can be found?
Sample Control: Defined as the ability of the method to reach the participants specified by the sample
definition. It can be broken down further into three useful components:
Theoretical Coverage: Is it possible, in theory, to reach all potential participants?
Sample Frames: To what extend do the lists you have available to you reduce the actual coverage?
Efficiency: Can the above two factors be mitigated by greater efficiency—the ease with which
qualifying participants can be found?
Quantity of data: How much information can be gathered will depend on the interest level the participant has
in the subject, how engaging the task is, and the ease with which the interview can be abandoned once
started.
Quantity of data: How much information can be gathered will depend on the interest level the participant has
in the subject, how engaging the task is, and the ease with which the interview can be abandoned once
started.
Response Rate The majority of non-response in interviewed methods is made up of refusal and ‘not-at-
home’. Refusal can be reduced by actions taken by an interviewer (persuasion and follow-up/re-
contact). Not-at-home is reduced by repeated contacts at different times and days. In non-
interviewed methods the reasons for non-refusal are merely conjecture, non-response is reduced
through use of reminders
Response Rate The majority of non-response in interviewed methods is made up of refusal and ‘not-at-
home’. Refusal can be reduced by actions taken by an interviewer (persuasion and follow-up/re-
contact). Not-at-home is reduced by repeated contacts at different times and days. In non-
interviewed methods the reasons for non-refusal are merely conjecture, non-response is reduced
through use of reminders
Situational factors related to the practical aspects of the project plus budget & time constraints
Situational factors related to the practical aspects of the project plus budget & time constraints
Environment: The situation the participant is in, both in terms of physical and psychological comfort and how conducive it is for concentrating on the survey and tasks.
Environment: The situation the participant is in, both in terms of physical and psychological comfort and how conducive it is for concentrating on the survey and tasks.
Field Force Control: How easy it is to monitor and check the work done by interviewers, either in the selection
of participants or in administering the survey itself.
Field Force Control: How easy it is to monitor and check the work done by interviewers, either in the selection
of participants or in administering the survey itself.
Interviewer Bias: The extent to which the interviewer, however unwittingly, conveys his or her own opinion
and thus suggests an answer which the participant is biased to agree with.
Interviewer Bias: The extent to which the interviewer, however unwittingly, conveys his or her own opinion
and thus suggests an answer which the participant is biased to agree with.
Speed and Cost: In data collection, speed is normally measured from the moment the survey questionnaire is
fully programmed and ready to be used for interviewing to the point where data processing and
tabulation can be done.
Speed and Cost: In data collection, speed is normally measured from the moment the survey questionnaire is
fully programmed and ready to be used for interviewing to the point where data processing and
tabulation can be done.
Participant factors related to the participants, who they are & how they will react to the survey
Participant factors related to the participants, who they are & how they will react to the survey
Perceived Anonymity: The extent to which the participant feels the interviewer or researcher will not know who they are.
Perceived Anonymity: The extent to which the participant feels the interviewer or researcher will not know who they are.
Social Desirability Bias: The tendency of participants to give answers that are socially acceptable whether or not they are true.
Social Desirability Bias: The tendency of participants to give answers that are socially acceptable whether or not they are true.
Sensitive Information: Surveys may require asking for information that participants consider highly personal, confidential, or otherwise sensitive.
Sensitive Information: Surveys may require asking for information that participants consider highly personal, confidential, or otherwise sensitive.
Low Incidence Rate: How hard it is to find a qualifying participant will affect both project speed and cost.
Low Incidence Rate: How hard it is to find a qualifying participant will affect both project speed and cost.
Participant Control: The extent to which the participant is in control of the survey-taking experience, when it
is done, and the speed at which it is completed.
Participant Control: The extent to which the participant is in control of the survey-taking experience, when it
is done, and the speed at which it is completed.
Quality control check: This should be reviewed as soon as possible. In face-to-face and telephone projects a proportion of participants may be re-contacted to verify that that the interview actually took place and to confirm key pieces of information.
In online projects quality control checks will often be built into the survey itself (i.e., checking for excessive speed).
Quality control check: This should be reviewed as soon as possible. In face-to-face and telephone projects a proportion of participants may be re-contacted to verify that that the interview actually took place and to confirm key pieces of information.
In online projects quality control checks will often be built into the survey itself (i.e., checking for excessive speed).
Fieldwork control: Review quota achievement and anticipate any problems that may impact timings for project delivery. If
possible, listen in to some interviews or accompany an interviewer into the eld. This nuance can be added to the analysis.
Fieldwork control: Review quota achievement and anticipate any problems that may impact timings for project delivery. If
possible, listen in to some interviews or accompany an interviewer into the eld. This nuance can be added to the analysis.
Partial data review: Review any questions that impact costs to client – typically incidence rates and interview lengths. Act
quickly should the need arise.
Partial data review: Review any questions that impact costs to client – typically incidence rates and interview lengths. Act
quickly should the need arise.
Passive Research
Within primary data collection, where the survey is designed specifically for the current research question, passive data collection covers all instances where no questions are asked and yet data is being deliberately collected.
Passive Research
Within primary data collection, where the survey is designed specifically for the current research question, passive data collection covers all instances where no questions are asked and yet data is being deliberately collected.
Trace analysis has long been used to impute data that could have been asked in a survey. Much of this data is fortuitous rather than
planned and divorced from survey methods
Trace analysis has long been used to impute data that could have been asked in a survey. Much of this data is fortuitous rather than
planned and divorced from survey methods
Examples of Trace analysis
Eye-tracking devices to measure what is being looked at. Pupilometers, Psychogalvanometers and Voice Pitch analyzers to measure emotional responses. Response latency to measure certainty.
Examples of Trace analysis
Eye-tracking devices to measure what is being looked at. Pupilometers, Psychogalvanometers and Voice Pitch analyzers to measure emotional responses. Response latency to measure certainty.
One well-developed passive measurement
technique is the use of cookies to track
online and/or mobile advertising exposure.
One well-developed passive measurement
technique is the use of cookies to track
online and/or mobile advertising exposure.
Data that can be collected passively from a mobile phone include such things as location, altitude, speed, temperature, network used,
number of calls made/texts sent, time of calls/texts.
Data that can be collected passively from a mobile phone include such things as location, altitude, speed, temperature, network used,
number of calls made/texts sent, time of calls/texts.
The major advantage of passive measurement is that it allows for the accurate collection of data that is either flawed in memory or obtrusive to capture in a diary study.
The major advantage of passive measurement is that it allows for the accurate collection of data that is either flawed in memory or obtrusive to capture in a diary study.
Currently costs may be high, since meters often need to be developed for each application and are not available ‘o the shelf’.
Currently costs may be high, since meters often need to be developed for each application and are not available ‘o the shelf’.
Online Research
Online research allows people to complete surveys over the Internet, using any suitable device
Online Research
Online research allows people to complete surveys over the Internet, using any suitable device
Mobile research is a subset of online research where the
device used to complete the
survey is a mobile phone.
Mobile research is a subset of online research where the
device used to complete the
survey is a mobile phone.
Participants in online surveys
are most often members of
online access panels, which are databases of potential survey takers set up expressly for the purpose of market research.
Databases of customers or prospects are also sources of sample.
Participants in online surveys
are most often members of
online access panels, which are databases of potential survey takers set up expressly for the purpose of market research.
Databases of customers or prospects are also sources of sample.
If the interviewer conducts the interview, using the Web
survey as their guide, then it is considered a Computer Assisted Personal Interview CAPI .
If the interviewer conducts the interview, using the Web
survey as their guide, then it is considered a Computer Assisted Personal Interview CAPI .
Mail Surveys
Mail surveys are pen and paper self-completion surveys.
Surveys may be addressed to individuals or to certain roles within the household
Mail Surveys
Mail surveys are pen and paper self-completion surveys.
Surveys may be addressed to individuals or to certain roles within the household
Kiosks
Kiosks are a form of self-completion survey, electronic in nature. The experience is similar to banking at an ATM machine.
The survey presented in a kiosk can be as simple as a single question; “How was your experience today?” with buttons to
push to indicate the answer, or as complex as a fully programmed survey with a touchscreen to capture the participant input.
Kiosks
Kiosks are a form of self-completion survey, electronic in nature. The experience is similar to banking at an ATM machine.
The survey presented in a kiosk can be as simple as a single question; “How was your experience today?” with buttons to
push to indicate the answer, or as complex as a fully programmed survey with a touchscreen to capture the participant input.
Indirect - Data that is collected without questions being asked.
Audits - Counting the prevalence of the subject matter, among a sample.
Passive Research - Data collected about a participant by use of a meter.
Quantitative Observation - Counting the prevalence of an event at a specific location.
Indirect - Data that is collected without questions being asked.
Audits - Counting the prevalence of the subject matter, among a sample.
Passive Research - Data collected about a participant by use of a meter.
Quantitative Observation - Counting the prevalence of an event at a specific location.
The term mobile market research is not a distinct data
collection methodology, rather a means of undertaking
existing data collection methods or facilitating new or better research approaches.
The term mobile market research is not a distinct data
collection methodology, rather a means of undertaking
existing data collection methods or facilitating new or better research approaches.
There is a shift away from landline telephones toward mobile devices as a communication preference. There has also been
dramatic growth in mobile only households
There is a shift away from landline telephones toward mobile devices as a communication preference. There has also been
dramatic growth in mobile only households
Face-to-Face: On-street
On-street interviews do not necessarily take place, as the name suggests, out in the street. People may be intercepted at
convention centers, airports, amusement parks, even at the beach! Participants may also be intercepted within a retail store
or a restaurant. On-street interviews might be better thought of as an in-person intercept interview.
Face-to-Face: On-street
On-street interviews do not necessarily take place, as the name suggests, out in the street. People may be intercepted at
convention centers, airports, amusement parks, even at the beach! Participants may also be intercepted within a retail store
or a restaurant. On-street interviews might be better thought of as an in-person intercept interview.
The data can be collected via pen and paper or using a laptop or tablet (known then as CAPI , Computer Aided Personal
Interviewing).
The data can be collected via pen and paper or using a laptop or tablet (known then as CAPI , Computer Aided Personal
Interviewing).
Assuming a good match between street and target population (for example any shopper type) this method can be extremely efficient.
Assuming a good match between street and target population (for example any shopper type) this method can be extremely efficient.
In this method the researcher has sampled the address and the interviewer visits the house to solicit an interview. The individual in the household may be selected by reference to quotas or randomly using a Kish grid or other mechanism. The interview itself takes place in the home of the participant.
In this method the researcher has sampled the address and the interviewer visits the house to solicit an interview. The individual in the household may be selected by reference to quotas or randomly using a Kish grid or other mechanism. The interview itself takes place in the home of the participant.
Made possible by the Internet, communities are a relatively modern hybrid technique utilizing both qualitative and quantitative methods. Also known as MROC (Market Research Online Community) the community is generally a closed, by invitation only, Website. The Website contains a number of tools that can be used for gathering opinions. These include, but are not limited to, polling, bulletin boards (forums) and chat rooms. The multimedia aspects of a Website (the ability to upload and view videos and images) makes the MROC an engaging and immersive experience.
Made possible by the Internet, communities are a relatively modern hybrid technique utilizing both qualitative and quantitative methods. Also known as MROC (Market Research Online Community) the community is generally a closed, by invitation only, Website. The Website contains a number of tools that can be used for gathering opinions. These include, but are not limited to, polling, bulletin boards (forums) and chat rooms. The multimedia aspects of a Website (the ability to upload and view videos and images) makes the MROC an engaging and immersive experience.
Choosing Between Quantitative
Methodologies
The choice of the most appropriate data collection method will be a trade-o between various factors dependent on the
project objectives, the client’s demands and constraints, and the real choices available to the researcher in his or her country.
Choosing Between Quantitative
Methodologies
The choice of the most appropriate data collection method will be a trade-o between various factors dependent on the
project objectives, the client’s demands and constraints, and the real choices available to the researcher in his or her country.
Incentives are not uncommon in quantitative market research and have become more common with the rise of online research.
Incentives are not uncommon in quantitative market research and have become more common with the rise of online research.
The Collins English Dictionary de nes incentive as a “motivating in uence; stimulus … serving
to incite to action” and reward as “to give (something) to (someone), especially in gratitude for a service rendered;
recompense”.
The Collins English Dictionary de nes incentive as a “motivating in uence; stimulus … serving
to incite to action” and reward as “to give (something) to (someone), especially in gratitude for a service rendered;
recompense”.
Audits Audits , as the name suggests, are a complete count of all instances of the subject of interest among a sample. Audits are typically used where consumer recall would be flawed and/or a demanding task.
Audits Audits , as the name suggests, are a complete count of all instances of the subject of interest among a sample. Audits are typically used where consumer recall would be flawed and/or a demanding task.
True or False : Internet penetration rates are very high in the top five research economies in the world.
True
____________ refers to what participants actually have to do during the research. These may be simple or complex sometimes requiring many instructions and even guidance by an interviewer.
Survey task
True or False: For multi-national research, you should consult resources such as the ESOMAR Global Market Research reports, national census data (if available), and other reputable sources to assist you in selecting the data collection methods.
True
When purchasing your sampling frame, such as a list for a telephone study, you should:
Purchase a bit more sample than you think the study will need (so fielding does not have to stop while waiting for more sample)
True or False: In every country in the world, landline telephone subscribers outnumber mobile phone subscribers.
False
True or False: Passive measurement techniques bring special ethical considerations for the market researcher.
True
Which of the following is NOT a good reason for a researcher to monitor phone or in-person interviews as they are being conducted?
To get away from other work and appear involved in the research
Quantitative research requires __________ of representative cases and __________ questions.
large numbers; structured
The POWER of all audit data (like a diary study in market research) resides in its __________.
completeness
What incentive would you recommend if conducting a 20 minute telephone interview with specialty doctors, such as general surgeons?
A sizable cash reward/charitable donation
Survey response rates are a function of:
Length of the survey Incentives offered Length of the fielding period Appeal of the invitation to participate (All the above)
What can a researcher do to make sure that survey quotas are working as intended? (SELECT ALL ANSWERS THAT APPLY.)
Test the survey giving responses that should assign you to different quotas and see where the program places each example.
Look at data from the first few completes, making sure the quota assignments match their responses.
Which of the following is NOT a quantitative data collection method?
Dyads or triads
These are the methods Paper survey by postal mail,
Mechanical scanners, Kiosks, A CATI study
Which one of these is NOT a passive measurement?
Recalled behavior
Your client suggests that, for a B2B project, your interviewers simply stand near the factory entrance and interview workers as they are arriving or leaving work. What should be your PRIMARY concern about this suggested method?
Interviewers can only stand in public spaces, outside the factory might be loitering or prohibited.
True or False : ONLY surveys completed using a PC or a laptop are online surveys.
False
Conducting quantitative research with a paper survey delivered by postal mail is currently _______ quantitative methodology and is _______.
the least utilized; very good at generating a representative sample
At what stage of the research process should you begin to consider the feasibility and cost of different data collection methods for a multi-national study?
At the proposal, bid stage
Which of the following is NOT a good reason for a researcher to monitor phone or in-person interviews as they are being conducted?
To get away from other work and appear involved in the research
The good reasons:
To add color to your understanding of the research findings
To check on the length of time the interview is taking
To hear if questions are being understood correctly & answered as intended
To ensure the initial, pretest interviews go smoothly before beginning full fielding
Please think about strengths of observational methods compared to survey methods. For which of the following objectives would observation provide MORE accurate data?
Determining how many people actually attend a sporting event
Face-to-Face: In-home interviewing has a number of weakness when compared to other data collection methods. Which is NOT a true weakness of this method?
Cannot collect as much information as other methods
When compared to survey methods, a DISADVANTAGE of observational methods of data collection is:
Underlying motivations, attitudes, and perceptions cannot be measured
____________ is the tendency of participants to give answers that are acceptable and conforming to expectations of good behavior, when the response is actually not true.
Social desirability bias
True or False: Passive measurement techniques bring special ethical considerations for the market researcher.
False
Researchers have many important considerations when selecting the optimal data collection method to use for a quantitative study. Which of the following is NOT a key consideration?
Whether they will be able to use the most common statistical analysis techniques
A company needs to obtain quantitative feedback, from a large sample of consumers, on a new video advertisement. Your client needs the fastest and least expensive methodology and is less concerned with projecting to the population. Which data collection method would BEST meet the needs of the project?
Online survey
Based on the amount of money spent on market research globally (source: ESOMAR’s Report), where does Quantitative research (as a whole) RANK when compared to qualitative and other types?
First: Most money spent
Which of these is a DISADVANTAGE of paper surveys sent by postal mail when compared to other methods?
Good perceived anonymity
If being conducted by an unskilled interviewer, which of the following methods has the HIGHEST potential for interviewer bias to effect results?
Diary study
A company needs to obtain quantitative feedback, from a large sample of consumers, on a new video advertisement. Your client needs the fastest and least expensive methodology and is less concerned with projecting to the population. Which data collection method would BEST meet the needs of the project?
Online survey
