Proctor 1 Exam DESIGN AND DATA/ QNR&MEASUREMENT DESIGNS Flashcards
This deck has QNR&MEASUREMENT DESIGNS and QUALITATIVE NOTES
What is Secondary research?
Defining the business problem or how to address It, however it needs to be carefully evaluated, since it was gathered for another business problem.
Example of Internal secondary research
prior research, customers database
Example of External secondary research
political articles, papers, syndicated research, government data
The business problem IS the problem confronting the clients
The business problem IS the problem confronting the clients
Business Problem:
- Ask what the decision makers need to know
- Action-oriented
- Focuses on symptoms
Market Research Problem
- Ask what information is needed and how it should be obtained
- Information oriented
- Focuses on the underlying causes
The information gathered to address the market research problem is used to take action to address the business problem
The information gathered to address the market research problem is used to take action to address the business problem
Business problem examples:
- Should a new product be introduced by a tech company?
- Should the advertising campaign of an innovative manufacturer be changed?
- Should the price of a laundry detergent brand increase?
Market research problem examples
- Determine consumer preferences and purchase intentions for the proposed new product.
- Determine the effectiveness of the current ad campaign
- Determine the price elasticity of demand and impact on sales and profit of various levels of price changes
The market research problem needs to allow the researcher to obtain all the information required to address the business problem and it guides the researcher throughout the rest of the market research process.
The market research problem needs to allow the researcher to obtain all the information required to address the business problem and it guides the researcher throughout the rest of the market research process.
Business problem can have two issues too broadly or too narrowly. What do both mean?
Too broadly: not specific enough to determine an approach and research design.
Too narrowly: all possible courses of action may not be defined.
Business Problem + Market Research Problem equal what ?
Research Objective
What are the terms Research Objective ?
- Goal oriented – questions that specifies what information is needed to solve a problem
- What researchers must do; have to be clean, specific, and actionable
- Designed to address both business and market research problem
- It’s the foundation of the research plan
Think of the market research problem as broad /general statement while research objectives are specific components of the market research problem.
Think of the market research problem as broad /general statement while research objectives are specific components of the market research problem.
Purpose of Research
- To prove a point/advance an interest of those conducting the research
- Can help identify limitation of using the research
- Hard to determine the true purpose – caution must be exercised
Reliability of the information
- Who conducted the research
* Quality, consistency, expertise, creditability are extremely important
Data collection Methodology
- Who was sampled, type of people, sample size, mode of collection (phone, online)
- Understanding the target population is adequately represented
Nature of the data collected
how questions were asked and analyzed
Accuracy of Secondary data
Its difficult; however you can verify with other research to compare results
Timeliness of the research
when the data was collected
Research Types
- Research evolves over time; qualitative was distinctly different from quantitative research
- Primary was different from secondary ; it was rare that a project could accommodate both descriptive and exploratory research
- As tools are becoming more sophisticated; lines are becoming blurred
Qualitative Research
- Smaller sample
- Flexible and unstructured
- Open-ended questions and probing
- Analysis is non statistical and subjective
- Tentative and Directional
- Meant for initial understanding; usually followed up with additional research
- Examples include: focus groups, IDIs, Observations, Ethnographies and online bulletin boards
Quantitative Research
- Large sample, statistically projectable
- Formal structure questions
- Clearly defined
- Statistical and objective
- Conclusive and definitive
- Findings are needs as input into decision making and reconvened course of action
- Examples include; online surveys, mobile surveys, telephones surveys, mail surveys, and in person interviews
Sequential
research methods followed by one another
Concurrent
research methods occur at same time
Embedded
research methods are within one another
Multi-level
any of the combined above
Exploration
written critical actions and solutions are unknown
Expansion
building and learning that was already collected
Triangulation
confirms that findings are valid and reflect the population
Transformation
paradigm shifts and changes thinking
Explanation
what is it and is it possible
Consumer intimacy
brings target audience to life and reinforce findings
Clients usually ask for data synthesis and a combination of both
Clients usually ask for data synthesis and a combination of both
What is Primary Research?
o Original research tailored to address specific business problems
o You control the research design
What is Secondary research?
o Defines the business problem but could also address the market research problem
o Data is collected for another purpose by someone else, public access or for purchase
Big data
is capturing, processing, integrating, and interpreting extremely large volumes of data originally from a variety of sources
In the business problem statement, it is important that the business problem be stated (worded) using the __________ perspective and language.
client’s / business’s
A common description of a business problem from clients is that market share is declining. To the researcher, this description is:
a symptom being caused by an underlying problem
True or False: The business problem statement must be formulated as a statement; it should NEVER be formulated as a question.
False
The FINAL step in defining the business problem so that the research questions can be considered is to:
Formulate the business problem statement
The FIRST step in a market research project is:
Defining and understanding the business problem
You are considering the sponsor of a secondary research study you are examining. Which type of sponsor should raise MORE concerns about the accuracy of the findings?
An organization interested in encouraging actions such as sales and investment
As a client or research buyer, you should strive toward communicating the __________ information about the business problem including relevant information, background, assumptions (and their confidence) in your discussions with the researchers.
maximum possible
When thinking about the differences between the business problem and the market research problem, which are characteristics of the BUSINESS problem statement?
- Action-oriented
- Focuses on symptoms
- Asks what the decision-makers need to know
Price changes, product improvements, promotion of any kind, and adjustments in channels of distribution are examples of:
Decision alternatives the business could take to address the problem
True or False: When managers must make decisions and they have inadequate information (or lack confidence in their information), this indicates a need for primary research.
True
Which BEST describes the overall purpose of the market research industry?
Assisting business decision making
Which of the following is NOT a part of a good research design?
Detailing all “nice to know” and extraneous information to be collected
Based on this course, the MOST serious mistake one can make during research design is:
Asking the wrong questions
True or False: The analysis plan for a study does NOT need to be considered until after a research proposal is accepted by a business.
False
__________ research designs are based on established theories, models, or best practices. They are often more efficient and allow for comparison across projects to normative databases.
Standardized
Examples of synergizing secondary data with primary data include which of the following? (SELECT THREE [3] ANSWERS THAT APPLY.)
In a study to be conducted with physicians, including their actual prescribing data from an external source in data analysis
Using secondary data available to address the business problem and planning to conduct exploratory research to better understand this information
In a study to be conducted with consumers in specific regions of a country, including measures from sales data and marketing efforts in each region for analysis
__________ uses a flexible and unstructured line of questioning, whereas, __________ is formal, structured, and specifies the precise information needed.
Qualitative research; quantitative research
Your client is interested in precisely measuring the optimal positioning, within the market, for their new product currently being developed. They have very little prior experience or background knowledge in this competitive product category. Identify the type of research MOST appropriate for this key objective.
Combination: Qualitative followed by Quantitative
__________ research is appropriate when you need to develop an initial understanding, which is often followed by other phases of research.
Qualitative
True or False: Causal research often uses an experimental design to control for the influence of potential “causes” on the “effect” being studied.
True
Market research proposals serve multiple purposes. Which of the following is NOT one of these purposes?
They always include a full, first draft of the survey / discussion guide.
Before researchers can work with a new client/business, they must often:
a. Sign non-disclosure agreements (NDAs)
b. Get a service agreement in place / meet the company’s standard operating procedures
c. Get onto the list “preferred research suppliers” of the business
d. Demonstrate (often in responses to RFIs) they have the expertise the company desires
***All of the above
A market research proposal should communicate what the project ___________ as it relates to the business needs the project will address.
does and does not do
Which of the following statements about how buyers (the client or business) evaluate market research proposals is NOT true?
All buyers are similar in how they evaluate every research proposal.
True or False: Research companies must be honest about their qualifications and abilities to conduct the project they are proposing.
True
How can researchers maintain the confidentiality of proprietary information of competing clients that the research company works for? (SELECT THREE [3] ANSWERS THAT APPLY.)
Business’s’ non-compete & confidentiality agreements
Research company confidentiality agreements and training for personnel
Separating researchers that work on competing company studies
When presented with a “research” project that is NOT true research (it is intended to further an agenda, sell, market…) the researcher should:
walk away
A researchers’ job begins when a client approaches you with a business problem. Your FIRST step in the market research process is to work with your client to learn about:
the challenges, opportunities, and information needs of the business
True or False: The purpose of interviewing industry experts before developing a research design is to better understand the business problem, its background, or context.
True
Reviewing secondary research can assist the business and the researcher by:
a) Helping to define the business problem
b) Providing some answers to the business problem, its background, or context
c) Refining the information needed to be gathered in primary research
d) Offering insights for the design of primary research (e.g., sample, question wording, analysis, etc.)
e) All of the above (this is correct)
Some distinctions are blurring in the market research industry. Identify the trends from this course that the market researcher should be aware of: (SELECT THREE [3] ANSWERS THAT APPLY.)
a) Primary research is increasingly utilizing secondary data to provide greater insights.
b) With widespread internet access, smart phones, etc., there is an increasing amount of secondary and meta-data available that may be useful to market research.
c) While still distinct in many ways, quantitative and qualitative research are becoming more intertwined within a broader project.
All of the following can be particularly useful when conducting a situation analysis EXCEPT:
Framing the problem so as to favor use of a particular research design
he more time and effort that has been spent on a business problem, the __________ difficult it can be to recommend that there is NOT a need for primary research to address the business problem.
More
Which of the following must be answered FIRST in order to develop an effective research design?
Why is the research needed and what are its goals?
Types of PRIMARY market research include: (SELECT THREE [3] ANSWERS THAT APPLY.)
a) Observation
b) Surveys/interviews
c) Focus groups/group discussions
True or False: Some market research projects include a qualitative and quantitative phase.
True
Which of the following are appropriate uses for qualitative research? (SELECT THREE [3] ANSWERS THAT APPLY.)
a. Develop hypotheses
b. Generate ideas
c. Develop a better understanding about a problem
True or False: Many quantitative studies use both descriptive and causal elements in their designs (together in the same study or as separate phases).
True
Which of the following is an example of how costs are often broken-out for different elements of the research design?
a) Sample & data collection
b) Incentives / honoraria / pass through expense
c) Sample size & analysis options
d) By assumptions such as incidence, interview length, etc.
e) All of the above (correct)
True or False: While the client usually includes the study background in their Request for Proposal (RFP), good researchers will add to the background from their own industry and research knowledge (including implications from secondary research available).
True
Which of the following statements about how buyers (the client or business) evaluate market research proposals is NOT true?
All buyers are similar in how they evaluate every research proposal.
True or False: In Requests for Proposals and during the proposal process, any constraints, restrictions, and deadlines should be stated clearly and up-front.
True
True or False: In discussions and in their Requests for Proposals (RFPs), clients need to disclose all relevant information about the project and not misrepresent the project.
True
One method to guide your situation analysis is the “Five Cs” approach. Which of the following is NOT one of the Five Cs?
controller
The 5Cs are company, competitors, customers, collaborators, climate
All of the following are essential to the process of defining the business problem EXCEPT:
Estimate costs to address the problem
What is essential is :
Understand the background of the problem, Formulate a business problem statement, Identify what information is needed, Determine what decisions the business needs to make
A new use for a product is identified. This is an example of what type of business problem?
Opportunity assessment
The background and context of the business problem will include:
Past forecasts Company resources and constraints Customer behavior changing Legal & regulatory environment All of the above (Correct)
Interactions between the researcher and the client/business should be characterized by all of the following EXCEPT:
Infrequent & brief interaction
The right interactions are openness, disclosure or relevant information and constraints, cooperation as a team, respect for each others time, infrequent and brief interaction
A researcher must assess the information available to address a business problem. In some cases, adequate information already exists. What should the researcher do in this situation? (SELECT TWO [2] ANSWERS THAT APPLY.)
Prepare a written explanation of why a primary research project is not needed
Discuss with clients how needs can be met with existing information
Indicators that primary research may NOT be appropriate or necessary to solve a business problem include: (SELECT THREE [3] ANSWERS THAT APPLY.)
Low-risk business decision must be made
The business has high confidence in the information it has available
The business’s directors and executives are never supportive of primary research results
How can a researcher express the value of market research to a business?
Demonstrating through discussions, research designs, proposals how business questions will be addressed
Providing needed deliverables such as tools, simulators, and working models
Skillfully helping to identify the business problem
TWO examples of standardized research designs, mentioned in this course, are: (SELECT TWO [2] ANSWERS THAT APPLY.)
Net Promoter Score
Van Westendorp Price Sensitivity
From this course, you know developing a good research design consists of answering the “5 Ws and 2 Hs” types of questions. Which of the following is NOT one of these questions?
To which syndicated study should we subscribe?
Based on this course, the MOST serious mistake one can make during research design is:
Asking the wrong questions
True or False; As research tools become more sophisticated, more research projects contain elements of both qualitative and quantitative as well as encompassing primary and secondary data.
True
Market research proposals serve multiple purposes. Which of the following is NOT one of these purposes?
They always include a full, first draft of the survey / discussion guide.
When the business problem is defined too __________, it can lack sufficient focus and the researchers not know where to start.
broadly
When the business problem is defined too __________, alternatives may not be adequately investigated or factors important to the problem may not even be considered.
narrowly
The concise description of the business problem or opportunity that may require market research is referred to as:
Business problem statement
Each of the following are sources of secondary research data and findings EXCEPT:
Conducting qualitative research yourself
What are
Internal company sales data
Prior research studies conducted by the business
Reports from government or industry associations
Results from syndicated research studies the business subscribes to
Which one statement BEST describes when a primary research project should be conducted:
When the cost of obtaining the information is less than the cost of making a bad business decision
Which BEST describes the overall purpose of the market research industry?
Assisting business decision making
Which of the following is a consideration when determining the research design for a business problem?
The clients’ resources available for the study (e.g., budget)
How information provided by the study will be used
Secondary research that informs the background, information needing to be collected, etc.
Timing needs for the research milestones
All of the above (correct)
A research design can BEST be described as:
“a plan of action for carrying out a research project”
Your client is interested in modeling a relationship (among many variables that could impact customer loyalty). Identify the type of research MOST appropriate for this key objective.
Quantitative: Causal
Which is a key assumption or specification that can have a major impact on a project’s total costs?
Sample size
Length of interview / amount of data collected
Incidence in the population/sample
Scope of analysis, reporting, presenting, & deliverables
All of the above (correct)
True and False: The client/business and the researcher need NOT be respectful of each other’s time since creating a thoughtful and effective research design and proposal requires only brief interaction.
False
A retailer wants to understand an increase in sales among baby boomers and a decrease in sales among millennials occurring during the past 12 months. This is an example of:
A business problem statement
All of the following are extremely important when evaluating secondary research EXCEPT:
Did the research utilize the most innovative statistical techniques?
When should the client and researcher consider whether primary research is truly needed to address the business problem?
As early as there are indicators that primary research may NOT be needed
After the researcher has defined the business problem, its background, decisions to be made, problem statement, and the broad research question, the researcher’s NEXT step is to:
Create objectives
Which of the following is typically included in the Background section of a proposal?
Sample design
Which of the “Five Cs” of situation analysis represents the legal and economic environment that impacts the business?
Climate
When there is not enough knowledge or understanding to proceed with a market research project, _____________ research is needed.
exploratory
A _____________ is typically issued to external research companies when a client/business has already designed the research including number and length of interviews, data collection method, and information to be obtained.
Request for Quotation (RFQ) / bid request
A thorough situation analysis with your clients about their business problem reveals they have a limited understanding of the problem. However, it is clear that a quantitative design will eventually be needed to address the need. What technique should you recommend NEXT to address this information gap?
Conducting qualitative, exploratory research
True or False: The purpose of interviewing industry experts before developing a research design is to better understand the business problem, its background, or context.
True
________ involves searching for and interpreting existing information, studies, and data relevant to the research topic.
Secondary research
Which of the following is typically included in the Background section of a proposal?
Context and environment of the business problem
Your initial situation analysis with your clients about their business problem is providing limited information. You specifically need to better understand their product’s advantages and disadvantages, in the context of competitors’ products. Which ONE of the following techniques will BEST address this specific need?
SWOT analysis
Before recommending a primary research project, you should consider each of the following EXCEPT:
The best sampling frame to use for the business problem
True or False: Causal research often uses an experimental design to control for the influence of potential “causes” on the “effect” being studied.
True
Conducting a __________ analysis is the key to understanding the background / context of the business problem and possible underlying causes.
situation
All of the following are essential to the process of defining the business problem EXCEPT:
Estimate costs to address the problem
Each of the following are sources of secondary research data and findings EXCEPT:
Conducting qualitative research yourself
In __________ research, questions are often asked in different ways and in a different orders to participants.
qualitative
Your clients desire a much better understanding of all the interactions a customer has with their company and its products, particularly those interactions that would be considered “pain points.” What type of research should you recommend?
A qualitative journey mapping study
Your client is interested in ideation, qualitative research to generate many new ideas for a new consumer product. What characteristics should your research design include to meet this objective? (SELECT TWO [2] ANSWERS THAT APPLY.)
Gather many participants together so more ideas may be generated & built upon.
Encourage participants to offer a large number of ideas without evaluating the ideas mentioned.
Research that focuses on gaining a holistic understanding of customers’ relationship with a brand or company, including all the “touch points” and “pain points” of contact, is called:
Customer experience (CX) research
True or False: Qualitative research, by itself, does not fulfill business needs. It must be used to inform and guide quantitative research in order for it to have value.
False
True or False: Qualitative observational research is based on the premise that individuals are not always able to articulate their innermost feelings about brands or products.
True
True or False: Focus groups are always to be preferred to In-depth Interviews wherever possible.
False
A concern in observational research is that:
Participants may NOT act as they normally would because they know they are being observed.
__________ qualitative research is when all participants and the moderator are engaged in the data collection process at the same time.
Synchronous
Message boards typically serve as the “hub” for __________ but they use other methods, such as polls, online chats, and other techniques, to keep members engaged.
online communities
True or False: A good qualitative moderator/interviewer will take minimal notes during focus groups/in-depth-interviews.
True
A qualitative research moderator/interviewer will ask follow up questions to:
Probe for clarity on what a participant said.
Probe to encourage a participant to say more.
Probe to gain a deeper understanding of a participant’s opinion.
Probe to ensure a participant has shared all of their ideas on a topic (none held back).
All of the above - the correct answer
Of the following steps in a qualitative study, which should typically be done BEFORE the others?
Finalize the screener so recruiting can begin
You are beginning an in-depth-interview study, consisting of 6 interviews on the first day. You MUST ensure that the desired number of interviews are conducted. What instructions should you give to the facility doing the recruiting?
Recruit 8 participants for 6 to show, provide reminders in advance, and recruit one “floater” to fill-in for no-shows.
Guidelines for conducting successful international qualitative studies include each of the following EXCEPT:
Avoid briefing foreign moderators on results from previous groups/in-depth-interviews (so that they have no preconceptions).
The right way; Begin the research in your home country., Respect recommendations of field agencies who are experts on research in their own country., Use native-speaking moderators/interviewers., Add extra time to the schedule in each foreign country for the unanticipated.
The most effective discussion guide for qualitative research will have each of the following features EXCEPT:
It will mainly include closed-ended questions intended to address business questions such as: What? Who? When? And How many?
The right way; It will reflect the agreement between the client and researcher about how to address objectives and what will be discussed., It should allow the moderator flexibility to follow the flow of the discussion and topics raised by participants (vs precisely following the guide)., It will rarely contain the term “Why,” but rather, use less confrontational language to encourage responses. , Its content will adapt to what has been learned and new questions that may emerge during the study.
When a participant in qualitative research is responding, a good moderator/interviewer is thinking about:
Both their response to the question being asked as well as how it relates to other questions that will be asked.
When qualitative research is underway, analysis and debrief sessions should focus on:
Old news (findings that confirm),
New learnings (Aha! Moments),
Changes that may be appropriate for the next day of research,
All of the above (the correct answers)
Qualitative research is ideal for hypothesis __________ while quantitative research is more appropriate for hypothesis __________ .
development; testing
Who’s responsibility is it to RECOMMEND the type of research (e.g., qualitative or quantitative, focus groups or IDIs, etc.) that best addresses the business need?
Research designer (research company)
What is the researcher’s BEST strategy when multiple stakeholders in a focus group study (marketing department, R&D, advertising firm, etc.) request more than can be accomplished effectively in one study?
Work with the direct client to help prioritize the objectives.
You need to understand how different hospitals go about their purchase process for radioactive isotopes (a highly specialized product). Neither you nor your client has background knowledge on the topic, which will be needed to inform the next phase of research. Which methodology would be MOST likely to address these objectives?
In-depth Interviews
True or False: Findings from a qualitative study can be generalized to a larger target population if care is taken in sampling and in statistical analysis of the results.
False
Markets for in-person qualitative research should be chosen PRIMARILY based on:
Business and sample-driven needs
In-depth interviews are BEST described as:
An experienced interviewer asking each participant unstructured questions in free-flowing manner
Selection of a videographer can be very important in certain qualitative studies. All of the following are important considerations for videography EXCEPT:
In ethnography studies, one stationary camera will meet all the video recording needs.
True or False: A contrived or artificial environment may be appropriate for observing participants when their behaviors do not yet exist in the natural environment or when it is not practical to observe the behavior otherwise.
True
An online message or bulletin board, which is open for days, weeks, or even longer, is an example of __________ .
asynchronous data collection
True or False: Screeners for qualitative research may be administered by traditional telephone interviewing or online via a short screening survey.
True
To obtain the best results possible in qualitative observation studies, the researcher should:
be unobtrusive/invisible so that participants forget they are being watched and act naturally.
True or False: Cash is the only type of incentive that should be considered for qualitative research participants.
False
A national, qualitative, in-person study will typically visit a minimum of __________ market(s).
two
For qualitative research, a recruiting organization will provide you with regular updates in the form of a “grid.” What do you need to monitor via these updates? (SELECT THREE [3] ANSWERS THAT APPLY.)
Is recruiting on-track to field the research?
Do adjustments need to be made to the screener to fill open slots in the schedule?
Do we have the right mix of participants scheduled?
True or False : A qualitative research moderator/interviewer is expected to word questions as they appear in the guide, just as an actor would speak their lines in a play as it is written in the script.
False
Which of the following is a qualitative indicator of what a participant is really thinking or feeling?
Voice inflection
Body language
Tone of voice
Speed of response
All of the above (correct answer)
True or False: When a moderator encounters “problem participants” in a focus group, best practice is to correct the behavior early in the group and in a compassionate and friendly way (without making the participant feel too self-conscious, unless it is unavoidable).
True
Decision trees and flow charts are reporting tools MOST applicable for which of the following qualitative research objectives? (SELECT TWO [2] ANSWERS THAT APPLY.)
Studies to understand a purchase process
Studies to identify all the touch points a customer has with a company or product
During qualitative data collection, our confidence in the research results increases when:
We begin hearing/observing the same things from participants.
As a client, a win-win solution to avoid one of your most relied-upon, independent moderators working for a direct competitor would be to:
Set up a retainer contract that assures the moderator a minimum level of work for your company in exchange for him/her not working for a competitor.
The reliability/repeatability of research results is GREATEST in __________ studies.
quantitative
Qualitative and quantitative research are similar in that they seek to:
minimize risks associated with business decisions
Your clients desire a much better understanding of all the interactions a customer has with their company and its products, particularly those interactions that would be considered “pain points.” What type of research should you recommend?
A qualitative journey mapping study
When possible (and practical), qualitative observational researchers usually prefer to observe participants in a __________ environment.
natural
Historically, most qualitative market research uses __________ because it is less time-consuming and expensive than __________ .
direct questioning; observation
The growth and popularity of smart phone technology (with audio, images, and video) is making qualitative __________ techniques more affordable and less time-consuming.
observation
Regardless of whether you are considering focus groups or IDIs, in-person qualitative research is MORE LIMITED than online and phone qualitative methods with respect to:
the geography/markets from which individuals can participate
Conducting qualitative research with individuals about sensitive topics can be challenging when participants worry about how others will react to what they share. Which of the following is the LEAST effective solution to address this challenge?
Conduct an online video focus group where participants do not disclose their names.
True or False: In qualitative screeners, we usually include those employed in advertising and the industry that our project is about, since their expertise may be provide additional insights to our research.
False
In order to design effective qualitative research, its study screener, and discussion guide, what does the researcher REQUIRE?
An understanding of the business need & objectives
In qualitative projects, business executives, physicians, floaters, and low incidence participants have at least one thing in common relevant to research design. Identify this commonality:
They typically require higher incentives to encourage participation.
A host/hostess at a focus group facility often plays several roles for in-person research. Which of the following is an example of these roles?
Greeting participants and confirming they are the person scheduled and are qualified.
Ensure each participant will not harm group dynamics (e.g., by being drunk or having severe personal hygiene issues).
Advise on who to excuse (based on observing behavior in the waiting room) if too many show up.
Consult with the moderator on whether to allow a late arrival to participate.
All of the above– (correct)
Which should you focus on when analyzing and reporting on qualitative research, such as focus groups or in-depth-interviews? (SELECT THREE [3] ANSWERS THAT APPLY.)
Consensus
Verbatim quotes
Mixed reactions (some feel one way, some another)
True or False: A good moderator will ONLY probe with follow-up questions when probing is specified in the discussion guide.
False
Your client asks you for MAJOR revisions to a qualitative study’s conclusions, in a way that is NOT supported by the actual findings. How should you respond? (SELECT THREE [3] ANSWERS THAT APPLY.)
Build a case politely explaining why the change is not appropriate and discuss with the client.
Invite the client to submit an addendum (under his/her name) to share a different point of view of the findings.
Remind the client that the report, under your name, must not be changed.
Which of the following provides the BEST understanding of “why” and “how” participants think and behave the way they do?
Qualitative
Some business issues are more exploratory in nature, requiring a broad understanding of a situation and introducing decision-makers to the general nature of the market. What type of research is BEST to meet these objectives?
Qualitative
Clients tell you their products suffer from being seen as old-fashioned and are looking for creative ideas and collaborative efforts to solve the problem. Which would NOT be an effective methodology to recommend?
Online survey
You are conducting analysis and reporting on qualitative research (in-depth-interviews) and want to communicate how frequently participants volunteered a response about a company’s “good service.” How should you present this finding in your report?
“Good service” was offered by 5 out of the 10 participants (without being asked about service quality).
True or False: Focus groups are always to be preferred to In-depth Interviews wherever possible.
False
You are designing qualitative research and your target population has a very low incidence. The research objectives would benefit most from cross-discussion among participants and being able to observe their non-verbal behavior. What qualitative methodology should you recommend?
Online focus group through video chat
Regardless of whether you are considering focus groups or IDIs, in-person qualitative research is MORE LIMITED than online and phone qualitative methods with respect to:
the geography/markets from which individuals can participate
Of the following steps in a qualitative study, which should typically be done BEFORE the others?
Finalize the screener so recruiting can begin
True or False: Screeners for qualitative research may be administered by traditional telephone interviewing or online via a short screening survey.
True
Why are interventions and stimuli used in qualitative research? (SELECT TWO [2] ANSWERS THAT APPLY.)
To change the pace of discussion and keep participants interested and engaged.
To help participants express their attitudes and perceptions that may be difficult to articulate otherwise.
Best practices for observation of a focus group include each of the following EXCEPT:
Have the moderator interact with only one key observer who can prioritize and filter feedback from others.
the best practices are
To keep observers focused (not distracted) have each focus on one participant so they can represent that person’s perspective in later discussions.
Ensure the moderator is sent notes, during the group, from each of the observers who has ideas and questions for the group.
Schedule two times during the group when the moderator will leave the room and consult with observers.
Hold debrief sessions with observers between groups or before the next day of interviewing to consider findings and any changes needed for the next group.
During qualitative data collection, our confidence in the research results increases when:
We begin hearing/observing the same things from participants.
Effective research design requirement
• Understand the decision making process and participants
• Become very familiar with the category
• Be knowledgeable and understand the context, lifestyle and competitor
• Be familiar with external event that may unfold that may have impact on product
• Understand unexpected business development opportunities
• Take consideration other research that been conducted
• Design research with end in mind
Effective research design requirement
• Understand the decision making process and participants
• Become very familiar with the category
• Be knowledgeable and understand the context, lifestyle and competitor
• Be familiar with external event that may unfold that may have impact on product
• Understand unexpected business development opportunities
• Take consideration other research that been conducted
• Design research with end in mind
Research design- a plan of action for carrying out a research project; it includes research methods, QNR/form design, sampling analysis, key success criterion, who and how the results will be communicated
Research design- a plan of action for carrying out a research project; it includes research methods, QNR/form design, sampling analysis, key success criterion, who and how the results will be communicated
5W’s an 2H’s
How much?, who will research be conducted?, who should we interview? What information do we need? Where should the information be gathered? Why the research is being done? When should the research take place?
5W’s an 2H’s
How much?, who will research be conducted?, who should we interview? What information do we need? Where should the information be gathered? Why the research is being done? When should the research take place?
When a client brings you a laundry list of information, your job is to recommend what cannot be accommodated within the scope of the project. It is important as a research, you be selective in what you are trying to achieve and not try too many objectives in a single project.
When a client brings you a laundry list of information, your job is to recommend what cannot be accommodated within the scope of the project. It is important as a research, you be selective in what you are trying to achieve and not try too many objectives in a single project.
Net promoter score: producing a clear measure of an organization performance through it customers eyes; the percentage of promoters minus the percentage of detractors
Net promoter score: producing a clear measure of an organization performance through it customers eyes; the percentage of promoters minus the percentage of detractors
Concept and use test: design to measure consumer reactions to an project; 2 survey: concept test and user usage experience
Concept and use test: design to measure consumer reactions to an project; 2 survey: concept test and user usage experience
Success metrics, actionable, focuses on what criteria needs to be met it makes the project successful, criteria and action to be taken
Success metrics, actionable, focuses on what criteria needs to be met it makes the project successful, criteria and action to be taken
Casual
• Test hypotheses are used when need an effect of change in one or more marketing variable
• Test the effects of one variable
• Test impact on specific market or variable
• Experimental/ non experimental
Casual
• Test hypotheses are used when need an effect of change in one or more marketing variable
• Test the effects of one variable
• Test impact on specific market or variable
• Experimental/ non experimental
Descriptive
• Measures and describe the current state of the market
• Who are the users? When do they use the product? Why aren’t they using it?
• Preplanned/structured
• Learn about current marketplace users, competitors, demographic of customers, etc.
Descriptive
• Measures and describe the current state of the market
• Who are the users? When do they use the product? Why aren’t they using it?
• Preplanned/structured
• Learn about current marketplace users, competitors, demographic of customers, etc.
Exploratory
• Gain insight, develop hypothesis, great for new topic research, usually first phase of multi-level project, flexible/versaltle design, best first step If business problem is vague or not well understood
Exploratory
• Gain insight, develop hypothesis, great for new topic research, usually first phase of multi-level project, flexible/versaltle design, best first step If business problem is vague or not well understood
Exploratory – unstructured techniques in order to develop hypotheses
Exploratory – unstructured techniques in order to develop hypotheses
Conclusive: highly structured, statistically representative sample to prove or disapprove hypothesis
Conclusive: highly structured, statistically representative sample to prove or disapprove hypothesis
Descriptive: describe a product or market associated among variables; descriptive
Descriptive: describe a product or market associated among variables; descriptive
Casual: conclusive research aims to collect data on casual relationships
Casual: conclusive research aims to collect data on casual relationships
Elements of an MR proposal
Cost: cost conducting the research
Background: contains relevant info about business problems and context about topic and secondary research
Business problem/purpose: includes the statement about business problem usually from business prospective
Research objective: the info the research will provide to address the business problem
Research Design: outline and provide rationale for that design
Deliverable : what the client will receive, data analysis
Reporting: type of report; ppt word doc, excel
Schedule: how long each phase of the project will be
Elements of an MR proposal
Cost: cost conducting the research
Background: contains relevant info about business problems and context about topic and secondary research
Business problem/purpose: includes the statement about business problem usually from business prospective
Research objective: the info the research will provide to address the business problem
Research Design: outline and provide rationale for that design
Deliverable : what the client will receive, data analysis
Reporting: type of report; ppt word doc, excel
Schedule: how long each phase of the project will be
RFP- request for proposal – the client states the business problem and research objectives and ask the research to provide the recommendation on the research design; it also serves as a sales tool, giving researchers opportunity to demonstrate their capabilities
RFP- request for proposal – the client states the business problem and research objectives and ask the research to provide the recommendation on the research design; it also serves as a sales tool, giving researchers opportunity to demonstrate their capabilities
MR proposal - a written statement of research designs and deliverables; it includes the business problems, research objectives, deliverables, cost, timing and schedule
MR proposal - a written statement of research designs and deliverables; it includes the business problems, research objectives, deliverables, cost, timing and schedule
Primary qual fills in info about consumer needs and perceptions
Primary qual fills in info about consumer needs and perceptions
Many times, a research project involves both qual and quant; quant typically goes first to develop the research design to develop the hypothesis and aid to the development of the qnr; the qual phase helps to guide the researcher in developing the research design
Many times, a research project involves both qual and quant; quant typically goes first to develop the research design to develop the hypothesis and aid to the development of the qnr; the qual phase helps to guide the researcher in developing the research design
Qualitative- deeper understanding of the business problem and underlying issues; quantitative- all about measuring and quantifying the data
Qualitative- deeper understanding of the business problem and underlying issues; quantitative- all about measuring and quantifying the data
Review of Marketing Research Process and Terminology
Step 1: Framing the Issue: the process starts when your client approaches you with a business problem
Step 2: Developing an approach : identifying hypotheses and formulating research questions
Step 3: Research Design: Developing a concrete plan/design ; best approach to achieve it
Step 4: Data collection: fieldwork/collection
Step 5: Preparing and analyzing data: analyze to address market research objectives and provide insight into business problems
Step 6: Prepping the report: provide the written report on presentation
Review of Marketing Research Process and Terminology
Step 1: Framing the Issue: the process starts when your client approaches you with a business problem
Step 2: Developing an approach : identifying hypotheses and formulating research questions
Step 3: Research Design: Developing a concrete plan/design ; best approach to achieve it
Step 4: Data collection: fieldwork/collection
Step 5: Preparing and analyzing data: analyze to address market research objectives and provide insight into business problems
Step 6: Prepping the report: provide the written report on presentation
Exploring the value of MR
The value/return on the investment that MR bring is that it reduces the risk in making decisions
Will provide input on ways to increase customer satisfaction
It will identify customers who will be most responsive to the new product introduction
MR should strive to be more consultive than technical
Exploring the value of MR
The value/return on the investment that MR bring is that it reduces the risk in making decisions
Will provide input on ways to increase customer satisfaction
It will identify customers who will be most responsive to the new product introduction
MR should strive to be more consultive than technical
Process of defining the business problem and developing an approach
Process of defining the business problem and developing an approach
Understanding the business decisions and underlying problem is meant to be address and is most important step. A business problem is the key to determining whether or not there is a need to conduct research. There are two categories to not conduct research failure to meet objective and identification of an opportunity. Failure to meet objective: what actually happen was not suppose to happen Identification of an opportunity: when clients are trying to identify potential opportunities
Understanding the business decisions and underlying problem is meant to be address and is most important step. A business problem is the key to determining whether or not there is a need to conduct research. There are two categories to not conduct research failure to meet objective and identification of an opportunity. Failure to meet objective: what actually happen was not suppose to happen Identification of an opportunity: when clients are trying to identify potential opportunities
Tracking studies- identify any problem that my occur
Tracking studies- identify any problem that my occur
The business problem clients describe are actually symptoms and not underlying problems; symptoms are signals of problems
The business problem clients describe are actually symptoms and not underlying problems; symptoms are signals of problems
A situation analysis – key to understanding the background of the business problem and cause of symptoms
A situation analysis – key to understanding the background of the business problem and cause of symptoms
5C’s – company, collaborators, climate, competitors, and customers
5C’s – company, collaborators, climate, competitors, and customers
Company- objectives, strategies, capabilities, resources, and constraints
Company- objectives, strategies, capabilities, resources, and constraints
Competitors- present and future initiatives;
Competitors- present and future initiatives;
Collaborators- agencies, partners, supplies, and distributors (PADS);
Collaborators- agencies, partners, supplies, and distributors (PADS);
Climate-legal and economic,;
Climate-legal and economic,;
Customers- buying behavior, demographic, needs and wants ;
Customers- buying behavior, demographic, needs and wants ;
When not to research; information is already available, money and personnel, timing is wrong to conduct MR, mgmt. is not supportive, business decision does not justify, actual business may provide a better answer, cost outweigh value
When not to research; information is already available, money and personnel, timing is wrong to conduct MR, mgmt. is not supportive, business decision does not justify, actual business may provide a better answer, cost outweigh value
The business problem STATEMENT- a concise description of problem and opportunities that business mgmt. is facing that requires to make a decision
The business problem STATEMENT- a concise description of problem and opportunities that business mgmt. is facing that requires to make a decision
Reliability refers to consistency of measurement.
A measurement instrument gives
reliable results if repeated measurements yield the same
result, assuming that the object that is measured has not
changed.
Reliability refers to consistency of measurement.
A measurement instrument gives
reliable results if repeated measurements yield the same
result, assuming that the object that is measured has not
changed.
Another type of reliability examines internal consistency.
Internal consistency examines responses to similar questions
in a survey to see if participants are answering in a consistent fashion.
Another type of reliability examines internal consistency.
Internal consistency examines responses to similar questions
in a survey to see if participants are answering in a consistent fashion.
Validity refers to the relevance of measurement, that is, whether a measurement is measuring what it is supposed to be measuring.
Validity refers to the relevance of measurement, that is, whether a measurement is measuring what it is supposed to be measuring.
How well a scale measures
what it is supposed to measure is criterion validity.
How well a scale measures
what it is supposed to measure is criterion validity.
Content validity or face validity means that people agree that the scale appears to be measuring what it should be
measuring. Content validity is necessarily subjective.
Content validity or face validity means that people agree that the scale appears to be measuring what it should be
measuring. Content validity is necessarily subjective.
In deciding question order, it is important to
remember that prior questions will in influence
responses to later questions.
In deciding question order, it is important to
remember that prior questions will in influence
responses to later questions.
For instance, open-ended questions are typically placed toward the beginning of the questionnaire.
For instance, open-ended questions are typically placed toward the beginning of the questionnaire.
“The funnel approach is a
strategy for ordering questions in a questionnaire in which the sequence starts with general questions that are specific questions from biasing general questions.”
followed by progressively specific questions in order to prevent
“The funnel approach is a
strategy for ordering questions in a questionnaire in which the sequence starts with general questions that are specific questions from biasing general questions.”
followed by progressively specific questions in order to prevent
In most cases, key questions occur at the beginning of the questionnaire and less important questions should occur later in the questionnaire. If your key measure is overall satisfaction, then you would want to place that question earlier in the survey.
In most cases, key questions occur at the beginning of the questionnaire and less important questions should occur later in the questionnaire. If your key measure is overall satisfaction, then you would want to place that question earlier in the survey.
Typical ordering of
questions include chronology ( rst, next, next) or by topic area (questions about patients, then questions about what drugs
are used to treat these patients and under what circumstances, then questions that focus in on the drugs themselves).
Typical ordering of
questions include chronology ( rst, next, next) or by topic area (questions about patients, then questions about what drugs
are used to treat these patients and under what circumstances, then questions that focus in on the drugs themselves).
QNR Order
Screener Open-Ended Questions General Questions Specific Questions Demographics Follow-Up
QNR Order
Screener Open-Ended Questions General Questions Specific Questions Demographics Follow-Up
The first step in the questionnaire design process is identifying the information needed.
Many times the research proposal includes a list of
question areas. This list may include general areas of
inquiry such as taste appeal, brand and product imagery, etc. or the list may look more like the questions to be asked. Either way, this list becomes the guide for questionnaire development.
The first step in the questionnaire design process is identifying the information needed.
Many times the research proposal includes a list of
question areas. This list may include general areas of
inquiry such as taste appeal, brand and product imagery, etc. or the list may look more like the questions to be asked. Either way, this list becomes the guide for questionnaire development.
Qualitative research uses unstructured
discussion or moderator guides rather than
structured questionnaires.
Qualitative research uses unstructured
discussion or moderator guides rather than
structured questionnaires.
The discussion guide outlines the topics
covered, and within each topic, specific
question areas.
The discussion guide outlines the topics
covered, and within each topic, specific
question areas.
A key difference between a
quantitative survey and a moderator’s
discussion guide is that in a quantitative
survey, the interviewer reads the question
exactly as worded. In contrast, a moderator
words questions to suit the context of the
group. A questionnaire provides a script
while a discussion guide is a general outline
of topic
A key difference between a
quantitative survey and a moderator’s
discussion guide is that in a quantitative
survey, the interviewer reads the question
exactly as worded. In contrast, a moderator
words questions to suit the context of the
group. A questionnaire provides a script
while a discussion guide is a general outline
of topic
Discussion guides include:
Topics to be covered and key areas that should be asked for each topic.
Order in which topics will be covered.
How long the moderator plans to spend on each section.
Description of materials presented during the discussion.
Description of activities.
Discussion guides include:
Topics to be covered and key areas that should be asked for each topic.
Order in which topics will be covered.
How long the moderator plans to spend on each section.
Description of materials presented during the discussion.
Description of activities.
Programming for
CATI systems traditionally involved using a proprietary programming language designed to meet the unique needs of questionnaires. Today, most systems, especially online survey systems, instead provide a visual user interface for
specification and configuration of the questionnaire.
Programming for
CATI systems traditionally involved using a proprietary programming language designed to meet the unique needs of questionnaires. Today, most systems, especially online survey systems, instead provide a visual user interface for
specification and configuration of the questionnaire.
Survey tools typically boast many dozens of “question types”, but these are typically variations on a more general type
Survey tools typically boast many dozens of “question types”, but these are typically variations on a more general type
Survey systems also vary widely in their support for mixed-mode surveys and mobile support. Some systems can be used to
program a survey for CATI, online fielding, and a proprietary mobile app, while many are limited to deployment through web
browsers.
Survey systems also vary widely in their support for mixed-mode surveys and mobile support. Some systems can be used to
program a survey for CATI, online fielding, and a proprietary mobile app, while many are limited to deployment through web
browsers.
Participant engagement is an alternative phrase for “getting people to
cooperate with survey invitations
Participant engagement is an alternative phrase for “getting people to
cooperate with survey invitations
According to other scholarly sources, “engagement” is a psychological
phenomenon that occurs when the mind is highly focused on a specific
task or activity.
According to other scholarly sources, “engagement” is a psychological
phenomenon that occurs when the mind is highly focused on a specific
task or activity.
If we can create this heightened sense of focus (resulting in a state of
maximum concentration) in a survey environment, we can yield higher
quality responses and “mindfulness.” T
If we can create this heightened sense of focus (resulting in a state of
maximum concentration) in a survey environment, we can yield higher
quality responses and “mindfulness.” T
As we talk about “gamification” in the research industry, we should be aware of what makes games appealing in
general. Interactive game manufacturers have long used data collection techniques that measure the level of “engagement”
being driven particular game-play activities.
As we talk about “gamification” in the research industry, we should be aware of what makes games appealing in
general. Interactive game manufacturers have long used data collection techniques that measure the level of “engagement”
being driven particular game-play activities.
This expanded notion of engagement will also drive respondent cooperation, particularly since satisfaction in one survey
experience is highly correlated with future participation.
This expanded notion of engagement will also drive respondent cooperation, particularly since satisfaction in one survey
experience is highly correlated with future participation.
Even though quantitative research uses closed-ended questions extensively, more often than not it also includes a limited
number of open-ended questions. Open-ended questions, usually referred to as open-ends, do not force participants to
choose among a set of responses, but ask them to respond in their own words.
Even though quantitative research uses closed-ended questions extensively, more often than not it also includes a limited
number of open-ended questions. Open-ended questions, usually referred to as open-ends, do not force participants to
choose among a set of responses, but ask them to respond in their own words.
Examples of open-ends are:
What do you like about this product?
Why didn’t you renew your insurance policy with Company X?
What suggestions do you have to improve the service you get from your cell phone company?
What is the rst brand of car that comes to mind?
Examples of open-ends are:
What do you like about this product?
Why didn’t you renew your insurance policy with Company X?
What suggestions do you have to improve the service you get from your cell phone company?
What is the first brand of car that comes to mind?
There are several benefits to open-ended
questions. Open-ends give participants the
opportunity to voice their opinions
There are several benefits to open-ended
questions. Open-ends give participants the
opportunity to voice their opinions
The question, “what is the first brand of car
that comes to mind,” is an example of an
unaided recall question.
The question, “what is the first brand of car
that comes to mind,” is an example of an
unaided recall question.
However, unaided awareness, where the
participant recalls one or more items rather
than simply acknowledging recognition (“aided” recall) is more useful in understanding which brands consumers think about first when making purchases.
However, unaided awareness, where the
participant recalls one or more items rather
than simply acknowledging recognition (“aided” recall) is more useful in understanding which brands consumers think about first when making purchases.
Instead of coding , researchers will sometimes conduct a verbatim analysis. This involves reading the responses line for line
and trying to summarize the main themes that emerge. They select a few verbatim to illustrate the range of points of view.
Instead of coding , researchers will sometimes conduct a verbatim analysis. This involves reading the responses line for line
and trying to summarize the main themes that emerge. They select a few verbatim to illustrate the range of points of view.
One way to create efficiency in coding open-ends is to provide a prelist of expected responses. This is sometimes referred to as precoding. The researcher lists expected responses to the open-ended question and the interviewer, in face-to- face or telephone based questionnaires, selects the appropriate category based on the participant's answer. The researcher also includes an "other" category and the interviewer records responses that do not t into the precoded responses as "other."
One way to create efficiency in coding open-ends is to provide a prelist of expected responses. This is sometimes referred to as precoding. The researcher lists expected responses to the open-ended question and the interviewer, in face-to- face or telephone based questionnaires, selects the appropriate category based on the participant's answer. The researcher also includes an "other" category and the interviewer records responses that do not t into the precoded responses as "other."
Unstructured (open-ended) questions
should play a relatively limited role in the
design of most quantitative questionnaires.
Unstructured (open-ended) questions
should play a relatively limited role in the
design of most quantitative questionnaires.
Observation techniques do not involve
direct questioning of participants.
Observation techniques do not involve
direct questioning of participants.
Mystery shopping requires observations and
recording of what is seen, heard and
experienced.
Mystery shopping requires observations and
recording of what is seen, heard and
experienced.
Usually when people think of scales in survey
research, they think of what we call non-comparative
scales.
Non-comparative scales vary in many ways including the number of points used. In the example in the previous section, we
used a 10-point scale, but scales can be 4-point, 5-point, 6-point, 7-point, and so forth
Usually when people think of scales in survey
research, they think of what we call non-comparative
scales.
Non-comparative scales vary in many ways including the number of points used. In the example in the previous section, we
used a 10-point scale, but scales can be 4-point, 5-point, 6-point, 7-point, and so forth
. An odd number of scale points has a neutral point in the middle, while an even number of scale points has no neutral point. In
an even-numbered scale, participants must choose between a positive and a negative end
. An odd number of scale points has a neutral point in the middle, while an even number of scale points has no neutral point. In
an even-numbered scale, participants must choose between a positive and a negative end
The Likert scale asks participants the degree to which they agree or disagree with a series of statements.
The Likert scale is commonly used, easy for participants to understand and is easy to administer with any of the data
collection methodologies (phone, in-person, online, mail).
The Likert scale asks participants the degree to which they agree or disagree with a series of statements.
The Likert scale is commonly used, easy for participants to understand and is easy to administer with any of the data
collection methodologies (phone, in-person, online, mail).
The semantic differential is another commonly used scale. Semantic differential scales have end-point labels that are
opposites of each other. Researchers commonly use the semantic differential in advertising and branding research to
determine the extent to which certain attributes are associated with the brand or influenced by the advertising.
The semantic differential is another commonly used scale. Semantic differential scales have end-point labels that are
opposites of each other. Researchers commonly use the semantic differential in advertising and branding research to
determine the extent to which certain attributes are associated with the brand or influenced by the advertising.
Nominal scales use numbers to designate an object, but the number has no particular meaning
Nominal scales use numbers to designate an object, but the number has no particular meaning
In an ordinal scale, the numbers indicate rank order. However, the intervals between the numbers are not necessarily equal.
In an ordinal scale, the numbers indicate rank order. However, the intervals between the numbers are not necessarily equal.
In an equal interval scale, there are equal distances between the values in each scale item.
In an equal interval scale, there are equal distances between the values in each scale item.
A ratio scale includes an absolute zero point. In a ratio scale, the numbers represent equal intervals, just as an interval scale.
A ratio scale includes an absolute zero point. In a ratio scale, the numbers represent equal intervals, just as an interval scale.
While studies have typically used only one mode of data collection such as phone, online or in-person, increasingly
researchers are employing multi-mode techniques.
While studies have typically used only one mode of data collection such as phone, online or in-person, increasingly
researchers are employing multi-mode techniques.
Closed-end questions, also known as structured questions, only allow certain answer choices to a question. These
responses can be yes or no, they can be multiple-choice
questions listing several options, or they can be scale
questions.
Another type of closed-end question is the multiple-choice question.
Closed-end questions, also known as structured questions, only allow certain answer choices to a question. These
responses can be yes or no, they can be multiple-choice
questions listing several options, or they can be scale
questions.
Another type of closed-end question is the multiple-choice question.
Multiple-choice questions eliminate the need for coding open-ended questions, are quicker to administer and can make
completing the survey more pleasant for the participant.
Multiple-choice questions eliminate the need for coding open-ended questions, are quicker to administer and can make
completing the survey more pleasant for the participant.
Another issue with multiple-choice questions relates to possible biases introduced by the order in which response options
are shown or read. To overcome this source of bias, researchers typically rotate the order of response options.
Another issue with multiple-choice questions relates to possible biases introduced by the order in which response options
are shown or read. To overcome this source of bias, researchers typically rotate the order of response options.
The way you
write instructions and frame questions will vary
depending on the data collection method. The main
difference to keep in mind when designing a question
is whether the survey is interviewer-administered
or self-administered.
The way you
write instructions and frame questions will vary
depending on the data collection method. The main
difference to keep in mind when designing a question
is whether the survey is interviewer-administered
or self-administered.
Questions with complex skip patterns and rotations are
best suited for a survey that is computer programmed, whether it is self-administered or interviewer-administered.
Questions with complex skip patterns and rotations are
best suited for a survey that is computer programmed, whether it is self-administered or interviewer-administered.
Comparative scales
compare two or more objects. Several commonly used comparative scaling questions are paired comparison scales, rank
order scales, constant sum questions, and Q-sorts. Comparative scales are ordinal scales rather than interval or ratio scales.
Comparative scales
compare two or more objects. Several commonly used comparative scaling questions are paired comparison scales, rank
order scales, constant sum questions, and Q-sorts. Comparative scales are ordinal scales rather than interval or ratio scales.
You can ask participants to compare the ads
on a series of attributes .
You can ask participants to compare the ads
on a series of attributes .
Rank order scales ask participants to rank a series of items on some dimension. As an example: rank the ice cream flavors
from most to least preferred. Assign a rank of “1” to your most preferred flavor, a rank of “2” to your next most preferred
flavor. Your least preferred flavor gets a rank of “4.”
Rank order scales ask participants to rank a series of items on some dimension. As an example: rank the ice cream flavors
from most to least preferred. Assign a rank of “1” to your most preferred flavor, a rank of “2” to your next most preferred
flavor. Your least preferred flavor gets a rank of “4.”
Ranking gives relative preference order, but does not provide absolute information on how participants feel about a product
or concept.
Ranking gives relative preference order, but does not provide absolute information on how participants feel about a product
or concept.
Constant sum scales ask participants to allocate a fixed sum such as 100 points among a set of choices
This type of constant sum question is often used in a concept or product test questionnaire. The questionnaire poses the
constant sum question before the new concept or product is shown to participants to get their pre-introduction purchase
behavior.
Constant sum scales ask participants to allocate a fixed sum such as 100 points among a set of choices
This type of constant sum question is often used in a concept or product test questionnaire. The questionnaire poses the
constant sum question before the new concept or product is shown to participants to get their pre-introduction purchase
behavior.
Another type of constant sum question asks participants to allocate attributes .
fixed number of points to indicate the importance of attributes
Constant sum questions offer several advantages. They encourage participants to make
trade-offs
Another type of constant sum question asks participants to allocate attributes .
fixed number of points to indicate the importance of attributes
Constant sum questions offer several advantages. They encourage participants to make
trade-offs
Q-sorts can be used when there are a large number of items for participants to evaluate, sometimes as many as 50 to 100
items.
Q-sorts can be used when there are a large number of items for participants to evaluate, sometimes as many as 50 to 100
items.
Maximum di erential scaling or MaxDi is another comparative scaling technique that has recently gained in popularity
because it is easy to administer online.
Maximum di erential scaling or MaxDi is another comparative scaling technique that has recently gained in popularity
because it is easy to administer online.
Over the last few years a variety of terms have been used by researchers when talking about ways of designing surveys for
mobile devices, for example, device agnostic, mobile friendly, mobile optimized and mobile first
Over the last few years a variety of terms have been used by researchers when talking about ways of designing surveys for
mobile devices, for example, device agnostic, mobile friendly, mobile optimized and mobile first
Mobile rst is a term that is frequently used to describe operationalizing online or multi-mode surveys so that the
characteristics of mobile devices determine the design decisions. However, there is no definitive definition of what this is. In
general, it means designing a survey that functions well on mobile devices and is suitable for other modes.
Mobile rst is a term that is frequently used to describe operationalizing online or multi-mode surveys so that the
characteristics of mobile devices determine the design decisions. However, there is no de nitive de nition of what this is. In
general, it means designing a survey that functions well on mobile devices and is suitable for other modes.
Assuming that the researcher has access to a suitable software system, the key design
issues are…
1. The length of the survey
2 Fitting questions onto the screen
3. Fitting the survey to the operating system
4. The required bandwidth
5. Paradata
Assuming that the researcher has access to a suitable software system, the key design
issues are…
1. The length of the survey
2 Fitting questions onto the screen
3. Fitting the survey to the operating system
4. The required bandwidth
5. Paradata
Some research on research (RoR) has shown that mobile surveys generally take longer than the same study completed on a
PC, particularly if the mobile survey is rendered the same on both the PC and the mobile device (as opposed to being re-
designed for mobile)
Some research on research (RoR) has shown that mobile surveys generally take longer than the same study completed on a
PC, particularly if the mobile survey is rendered the same on both the PC and the mobile device (as opposed to being re-
designed for mobile)
Paradata is a term that describes data about the process by which survey data were collected, for example, the operating
system, the browser used, the screen size, the connection speed, elapsed time for the entire survey or even for each
question, and for participants who break o without completing the entire survey, the last question answered.
Paradata is a term that describes data about the process by which survey data were collected, for example, the operating
system, the browser used, the screen size, the connection speed, elapsed time for the entire survey or even for each
question, and for participants who break o without completing the entire survey, the last question answered.
Paradata is often very useful for diagnosing problems with the survey.
Paradata is often very useful for diagnosing problems with the survey.
In drafting your survey questions, you should always
consider whether the typical participant in your
sample would be able to answer your questions.
You also want to make participants feel comfortable
providing information about sensitive topics.
In drafting your survey questions, you should always
consider whether the typical participant in your
sample would be able to answer your questions.
You also want to make participants feel comfortable
providing information about sensitive topics.
If you are creating a new survey, specifically to be used for cross-cultural comparison, the best approaches involve
eliminating traditional rating scales and replacing them as appropriate with these:
Binary Scales, Choose many question, Ranking Questions, maxdiff scaling.
If you are creating a new survey, specifically to be used for cross-cultural comparison, the best approaches involve
eliminating traditional rating scales and replacing them as appropriate with these:
Binary Scales, Choose many question, Ranking Questions, maxdiff scaling.
When conducting international research, which type of scale is more appropriate?
Binary scales and comparative scales
Which of the following is NOT an open-ended or unstructured question?
Would you say price is unimportant or important to you in deciding what car to buy?
The questionnaire design process cannot begin until:
The information needs are defined
Which of the following is NOT a reason that a researcher would develop an observation form for observers watching shoppers purchase soft drinks in a grocery store?
Provide a script to observers for how to frame questions asked of shoppers
Which of the following is NOT an approach to assessing reliability?
Content reliability
While many survey systems can import questionnaires from text documents, what is typically lost?
Skip patterns
What design issues must be considered for online surveys taken on mobile devices?
The length of the survey
Fitting questions onto the screen
The required bandwidth
The paradata collected
All of the above (correct)
Which statement is TRUE of comparative scales?
The major benefit of comparative scales is to detect small differences.
Which type of scale has adjectives at each end of the scale that are opposites of each other?
Semantic differential
Pretests are useful because they _____________.
identify problems with questionnaire wording or question flow
Which of the following is NOT an example of a comparative scale?
Likert scale
Which of the following is a reason to provide a “don’t know” option for rating scales?
Forcing participants to answer questions they don’t know irritates them.
Forcing participants to answer questions they don’t know risks non-accurate responses.
Participants without an opinion may simply mark the middle scale point which may distort results.
In situations where some participants are expected to not have opinions or not to know, accuracy of the data can be improved by providing a “don’t know” option.
All of the above (correct)
A researcher must decide to remove some questions from a questionnaire because the interview is too long. Which question would be the BEST candidate to remove from the questionnaire?
The question provides new information because it has not been asked in previous surveys
For which data collection method is it most important to include simple yet detailed instructions for participants?
Mail survey
Which scale or scales have an absolute zero point?
Ratio only
_____________ is being used for measurement if participants are given 100 attitude statements on individual cards and asked to place them into 11 piles, ranging from “most highly agreed with” to “least highly agreed with.”
Q-sort
Is it a luxury or a necessity to pilot test a questionnaire?
Necessity
When trying to overcome participants’ unwillingness to answer, the researcher might list possible responses. The researcher provides this list in order to _____________.
reduce the effort required of participants
Which of the following is a disadvantage of a Likert scale?
Participants often acquiesce, signaling agreement even when they disagree.
True or False: Participants in some cultures may not use all of the choices offered by a scale.
True
Which statement is TRUE of comparative scales?
The major benefit of comparative scales is to detect small differences.
Which of the following is NOT an approach to assessing reliability?
Question options:
Content reliability
Which of the following drives the information needs for a questionnaire?
Business and research objectives
