Module #3 - Research Design Flashcards
Define “Research Design”
A FRAMEWORK for conducting the research project, including the details of the information sought, how it will be sought, and how it will be analyzed.
What is the basic difference between primary and secondary research?
Primary research is conducted specifically for the project at hand. Secondary research is existing data that is re-purposed to answer your current questions.
Name 3 common methods of data collection for primary Research
1) Survey/Individual Interviews,
2) Focus Groups
3) Observation
What 5 ways that locating and reviewing any existing secondary research help you before beginning your project?
It can help you: define your issue more clearly, develop your approach, identify key and relevant variables, test hypotheses and interpret your primary data more insightfully.
What are the four key considerations when evaluating the usefulness of secondary data?
Timeliness - is it recent enough to still be valid?
Relevance - how closely does it address the business questions you’re looking at?
Completeness - Does it cover everything you were looking to investigate? Is something missing?
Quality - You may not know the details of how the research was conducted, thus you can’t know if the data is valid.
Name the 3 broad categories of primary research design.
1) Exploratory
2) Descriptive
3) Causal
Describe “Exploratory Research”
PRELIMINARY research used to GAIN INSIGHT into the market or business issue, especially when tackling a NEW TOPIC - one where there has not been a lot of other research. However, even for well-researched topics such as laundry detergents, exploratory research can yield new insights or hypotheses. Very often exploratory research is the first phase of a multi-phase research project
Describe “Descriptive Research”
Research that DESCRIBES the CURRENT state of the market - DEMOGRAPHIC characteristics of current users, BRAND ATTITUDES and perceptions, CUSTOMER SATISFACTION.It is the most common research type and answers basic questions such as who are our users, when do they use our product, why are they using it or not using it, and where do they buy or use our product.
Describe “Causal Research”
Research that seeks to find out about CAUSE AND EFFECT RELATIONSHIP BETWEEN VARIABLES. Researchers use causal research when they want to assess the EFFECT OF A CHANGE IN ONE OR MORE OF THE MARKETING VARIABLES. For example, what will happen to sales if we change the price of the product? Will consumers be more attracted to our product if we change the packaging design? In causal research, we test the effect of one variable such as price on another variable such as sales
Which of the 3 types of research design are considered “conclusive”?
Descriptive and Causal
Which of the 3 types of research design are NOT considered “conclusive”?
Exploratory
“Descriptive” research answers which questions?
Who, What, Where, When and How
“Causal” research answers which question?
Why
Describe and draw the breakdown of Descriptive Research types.
Split into Cross section and Longitudinal.Then split Cross Section into Single Cross section and Multiple cross section
Name 3 types of EXPLORATORY Primary research
1) Expert Interviews
2) Focus groups
3) Pilot surveys
What type of research design is used when you want to explain, monitor or test hypotheses?
Descriptive research
What is the difference between cross-sectional and longitudinal designs?
Cross-sectional deals with a single point in time, and longitudinal deals with 2 or more points in time
What is a single cross-sectional design?
A single sample at a single point in time
What is a multiple cross-sectional design?
Two or more surveys conducted at 2 or more points in time (e.g., many tracking studies or pre- and post-tests).
What is a longitudinal design?
Same sample tested at multiple points in time, often using a panel.
What is the main advantage to using a longitudinal or panel design?
To be able to assess changes at the individual level. Multiple cross-sectional allows for assessing changes at the group level only.
Name 3 advantages of a cross-sectional design over a longitudinal design.
1) more representative (panelists are, by definition, not)
2) less response bias from prior surveys
3) speed and flexibility in data collection
Name 3 advantages of a longitudinal design over a cross-sectional design.
1) can detect change at respondent level
2) greater accuracy in reporting actual usage and behavioral data (panelists asked to record usage in real time)
3) Ability to collect more data.
What is the key disadvantage of multiple cross sectional vs. longitudinal.?
Cannot detect changes at the respondent level
Name 3 disadvantages of a longitudinal design.
1) not representative because people who agree to be in a panel are different from those who don’t, and respondents drop out from wave to wave.
2) Panelists are paid, thus are “professional” respondents.
3) The act of taking the survey may change the responses of some respondents in later surveys; response bias.
In causal research, which type of variable do marketing management elements (e.g., price, packaging, positioning) tend to be? Dependent or independent?
Independent
In causal research, which type of variable do business outcomes (e.g., sales volume, market share) tend to be? Dependent or independent?
Dependent
Why are experimental designs good for causal research?
They are an efficient way to infer cause and effect
How can you conduct causal research if an experimental design is not possible or advisable?
You can use a non-experimental design and infer cause and effect through statistical methods
What are the two classes of experimental design?
Field and laboratory
What is another term for a “Field experiment”?
In-market test or test market
What does a lab experiment attempt to do?
Simulate the real world, but in a controlled environment
What is another term for lab experiment?
Simulated Test Market
What are the 5 main drawbacks to field experiments?
1) Costly
2) Time intensive - takes time to set up and to wait for actual sales to occur
3) Cooperation - often difficult to obtain cooperation from the necessary parties, e.g., retailers, franchisees, etc.
4) Difficulty - lack of control over real world elements
5) Competitive concerns - exposing plan to public alerts competition, which can launch a response
What are the 5 key advantages of laboratory experiments?
1) quicker
2) cheaper
3) can control all/most aspects
4) don’t need cooperation from other parties, i.e, franchisees, distributors,
5) don’t risk revealing your plans to competition
What is the main criticism of laboratory research?
Environment is artificial and lacks realism, which can affect the external validity of the experiment
What is external validity?
The degree to which the results of the experiment can be generalized to the real world.
What is it called when respondents are sensitized to looking more closely to the experimental detail?
Demand artifact
Besides a potential lack of realism, what is another drawback of laboratory experiments?
Demand artifact, which is when respondents are conditioned by the circumstances of the experiment to look more closely at some details than they normally would.
What are extraneous variables?
They are all variables not manipulated by the experiment but that may exert influence on the dependent variable under study.
Why is it important to build in controls to minimize extraneous variables?
Because uncontrolled extraneous variables might call into question the validity of the experiment.
What are 3 common ways of controlling for extraneous variables?
1) Matching
2) Randomization
3) Statistical control, which means measuring extraneous variables and adjusting for their effects
What is “matching” and how does it help control extraneous variables?
Matching is the organizing of experimental units into groups so that they share some particular characteristics that are relevant to the research. A procedure designed to ensure that respondents or stores within a panel (those receiving a given stimulus) are as similar as possible to those in other panels.
What is the major drawback to “matching”?
Impossible to perfectly match two groups, and there will still be extraneous variables that affect the test group differently from the control group
What is randomization?
It is the random assignment of respondents to the test and control groups.
How does randomization help control for extraneous variables?
With randomization, extraneous variables are represented equally within each group.
What is the preferred method for controlling for extraneous variables in experiments?
Randomization
When is randomization less effective?
When sample sizes are small
Why is randomization less effective when sample sizes are small?
Because randomization only produces groups that are equal ON AVERAGE, rather than truly equal.
How can you check whether randomization has been effective?
By measuring the extraneous variables and comparing them across groups.
With respect to extraneous variables, what is “Statistical Control”?
It is measuring extraneous variables and adjusting for their effects with statistical analysis.
What is “random sampling error”?
The error due to the particular sample selected being an imperfect representation of the population of interest. It can be thought of as the variation between the true mean value of the sample and the true mean value of the population.
What is “non-sampling error”?
Errors that can be attributed to sources other than sampling.
Which kind of error will a good research design strive to minimize?
Non-sampling error
What are the two main categories of non-sampling error?
Non-response error and response error
What is “non-response error”?
It is the error that arises from the fact that there is a certain level of non-response in any survey methodology. If non-responders differ from responders in important ways then you have “non-response” error.
What is “response error”?
A type of non-sampling error arising from respondents giving inaccurate answers or having their answers misrecorded or misanalyzed.
What are the three main types of “response error”?
1) Measurement error
2) Recording and Analysis error
3) Respondent error
What is “measurement error”?
It is the error caused by the survey instrument, either in the way it is administered or the measurement itself.
Poorly worded questions are an example of what kind of error?
Measurement error
Name two sources of measurement error
1) Poorly worded questions that cause respondents to misinterpret its meaning
2) Interviewers misreading a question
How can a researcher minimize measurement error?
1) developing clear, well structured surveys
2) having rigorous interviewing standards
3) clear instructions for respondents in the case of self-administered surveys
What can a researcher do to catch and correct possible measurement error before launching a study?
Conduct pre-tests
How does conducting a pre-test help a researcher?
It helps catch and correct possible measurement error.
What is “recording and analysis error”?
It is the error that arises from interviewers making mistakes in recording responses, or from data processing or data analysis errors.
What is “respondent error”?
It arises when respondents provide misleading information, either deliberately or inadvertently.
What are two common reasons that respondents may deliberately provide misleading information?
1) Not wanting to admit to some behaviors
2) Overstating the degree to which they engage in socially desirable behaviors
Why might a respondent inadvertently provide misleading information?
Because the question is not clearly worded.
Qual v Quant: How do they differ in terms of sample size?
Qual uses small samples; Quant uses large samples
What are 3 key reasons for doing qualitative studies?
1) develop hypotheses
2) Gain in-depth understanding of and reactions from consumers
3) generate new ideas
What are four circumstances that call for doing quantitative studies?
1) provide sizing for targets, product demands, and confirm hypotheses
2) Measure differences between sub-groups
3) Determine the impact of alternative pricing, packaging, posititioning, etc
4) Assess ad recall, measure ad effectiveness
How does the measurement tool differ for qualitative studies vs. quantitative studies?
Qualitative uses semi-structured discussion guides, and quantitative uses structured questionnaires.
Is qualitative research cheaper and faster than quantitative?
Not necessarily
What is an “RFP”?
Request for Proposal, which will include recommendations for research design
What is an “RFQ”?
Request for Quotation, which will include costs and timing on a study that is already designed.
What is an “RFI”?
Request for Information, which will include information on the supplier’s capabilities, and strengths
What is an MSA?
Master Services Agreement
What are the 7 major pieces of a research proposal?
1) Background
2) Problem Definition/ Overall Purpose
3) Research Objectives
4) Research Design
5) Plan for Reporting (Deliverables)
6) Schedule/ Timeline
7) Costs
How should the problem definition be written?
From the perspective of the business issue
What do the research objectives do?
They state the information that the research will provide to address the business issue
What elements of research design are included in a proposal?
1) The research approach - qual/quant, special analytical models
2) The sample, including qualifications, sources, & size
3) Data collection method
4) Information about the questionnaire/survey instrument (outline, special Q techniques, length of interview
