Exam 2 Flashcards
How do you calculate Total Error?
Difference between the true value and the observed value of a variable.
How do we determine sample size using different sampling procedures?
Difference between Field versus laboratory experiments.
Field Experiments: Research study where one or more independent variables are manipulated by the experimenter under controlled conditions.
Laboratory Experiments: Experiments where the experimental treatment is introduced in an artificial or lab setting.
Pro’s / Cons of Field vs Lab experiments.
Pro’s vs Cons: (Lab vs Field)
* Low External validity vs. High External Validity.
* High Internal Validity vs. Low Internal Validity.
* High Control vs. Low Control.
* Low Cost/Time, vs High Cost/ Time.
* Low Exposure to Competition vs. High Exposure to Competition.
* Subjects are aware of participation vs. Subjects are unaware.
Threats to internal validity in experiments – 7 threats
Maturation, Instrumentation, Selection Bias, Mortality, History, Testing, Statistical Regression
Internal versus external validity in experiments
Internal Validity:
o Ability of the experiment to show relationships unambiguously.
o Whether the manipulation of the independent variables causes the effect
External Validity
o Whether the experiment results can be generalized beyond the experimental setting.
o Applicability of the experiment results to situations outside of the actual experimental context.
Understand the 3 conditions necessary for causation: correlation vs causation.
1: Condition of Concomitant Variation – There must be evidence that a strong association exist between an action and an observe outcome.
2: Condition of Time Order Occurrence – Must be evidence that the action preceded the outcome. (X before Y).
3: Absence of Other Causal Explanations: There must be evidence that there is no strong competing explanation for the relationship. That a high level of validity exists.
Lay out of questionnaire design. (9)
1: Cover letter
2: Consent Form
3: Screening,
4: Prompts / Questions order
5: Test / Controls
6: Attention Check
7: Sensitive questions near the end
8: Demographic questions.
9: Thank you note.
What is pretesting and why do you do it?
Pretesting refers to the testing of the questionnaire on a small sample of respondents to identify and eliminate potential problems.
This helps us ensure our questionnaire gathers the correct type of data we are looking for. It also helps identify problems that may be present.
Know differences between open-ended, close-ended, leading, double-barreled, loaded questions
Leading Questions try to influence behavior or certain answers.
Loaded Questions make assumptions about the respondent in the question.
Double Barrel Questions are when you compare/ask 2 or more questions at the same time.
What words and types of sentences should be avoided in a questionnaire?
Avoid complexity, ambiguous wording, leading questions, loaded questions, double barrel questions, assumptions, and burdensome questions
Open Ended Questions:
Allows respondents to answer in their own words.
Solicit recall information when research don’t want to be bias by listing alternative choices.
Helps identify possible response category options when the researcher is unable to anticipate range of responses that could exist.
Pros: Answer is in respondents own words, provides insight, allows for probing and additional alternatives.
Cons: Difficult to interpret, editing and coding can be a challenge, potential interview bias, lower responses, possibility of shallow answers
Close ended questions
Respondents are given a finite number of responses to chose from.
Two main ways to ask these kinds of questions: Either choose from a list of responses or do a single choice rating scale.
Pros: Easy and accurate data coding / entry, limited responses, alternative list may help respondents recall, limited interview bias.
Cons: Researchers must generate alternatives; respondents are forced to choose no freedom in answers for respondents
What are the 5 characterizations of scale and their definitions?
Generalizability: The ease of administration and interpretation in different research settings and situations.
Sensitivity: The extent to which ratings provided by a scale can discriminate between the respondents who differ.
Validity: Does the study measure what it is supposed to measure?
Reliability: The consistency with which the measure produces the same results with the same or comparable population
Relevancy: Refers to how meaningful it is to apply the scale to measure a construct. Mathematically, relevancy=reliability*validity.
What are the factors that affect Researcher’s choice of attitude scale?
Specific information, data collection method, Budget constraints, respondents’ attitude, and knowledge
Five issues in designing single item scales.
- Types of poles used in the scale,
- Number of scale categories.
- Strength of the anchors.
- Balance of the scale.
- Labeling of the categories
What are the 5 single Item Scales:
Itemized, Pictural, Rank Order, Paired Comparisons, and Constant Sum
Define what each non-comparative single item scale does: (2)
Itemized Respondent selects from a limited number of categories. Most used single item scale. Measures overall satisfaction.
Pictorial: Commonly uses Pictures to describe feeling. Format must be comprehensible to respond and allow accurate response.
Define what each comparative single item scale does: (3)
Rank Order: Respondents compare two or more items and rank them. The respondent should have knowledge of all options. The options must cover all possible choices. Order of the choice may affect the result.
Paired Comparison Scales: Ask respondents choose one of the two items in a set based on a specific criterion or attribute.
Constant Sum Scales: Ask the respondent to divide a given number of point, typically 100, among two or more attributes, based on the relative importance.
Define Multi-Item Scales
scales that measure a sample of beliefs toward the attitude objects and combines the set of answers into an average/sum score to construct an estimate of some underlying or abstract variable.
What do the 3 Multi-Item Scale do?
Semantic differential Scale – Used to describe the set of beliefs that comprise a
person’s image of an organization or brand.
- Likert Scale - Respondent specifies a level of agreement or disagreement with
statements express either a favorable or an unfavorable attitude toward the
concept under study. - Thurstone Scale – Also called equal appearing intervals, strength of the individual
items is taken into account in computing the attitude score
Definition of scaling.
Scaling is the Process of creating a continuum on which objects are located according to the amount of the measured characteristics they possess.
Qualitative Method Characteristics.
Used to understand human behavior from the informant’s perspective. Assumes a dynamic negotiated reality.
Data is collected through participant observation and interviews.
Data is analyzed by themes from descriptions of information.
Data is reported in the language of the informant.
Quantitative Method Characteristics:
Concerned with discovering facts about social phenomena.
Assumes a fixed and measurable reality.
Data collected through measuring things.
Data analyzed through numerical comparisons and statistical inferences.
Define Attitude:
Mental state used by individuals to structure the way they perceive their environment and guide the way they respond to it.
What is a construct?
an abstract idea specifically invented for a given research and/or theory building purpose. Typical marketing constructs are brand loyalty, satisfaction, preference, awareness, knowledge.
What are the 4 general scales? (NOIR)
Nominal - Objects are assigned to mutually exclusive, labeled categories – Count, percentages
- Ordinal - Ranks objects or arranges them in order by some common variable – median , mode
- Interval - Numbers used to rank objects also represent equal increments of the attribute being measured, differences can be compared, All statistical operations
- Ratio Scale - Possible to say how many times greater or smaller one object is than another, Only scale that permits comparisons of absolute magnitude
What reason may there be for a nonresponse error?
It can either be a phenotypic or Genotypic source.
Phenotypic includes characteristics of the data collection process, such as how the questions are asked, and how long the interview is.
Genotypic includes indigenous characters, such as their age, sex, or occupation.
What are the 4 types of errors in surveys?
Nonresponse errors due to refusals
Inaccuracy in response
Unwillingness to respond accurately
Interviewer error
Ways to improve response rate
Use incentives like cash, or noncash items.
Follow-up your contacts.
Send surveys through the mail.
Incentives, promised anonymity, length
and questionnaire design, less intrusive questions
Define the response rate?
Response rate in surveys refers to the number of people who answered the survey, divided by the number of people in the sample.
Define Survey
- Survey is the overwhelming choice of researchers for collecting primary
data.
What are the 5 personal interview techniques?
Purchase Intercept, Executive Interview, Mall Intercept, Omnibus, Door-to-Door interviews
Pro’s and Con’s of Survey
Advantages – Speed, accuracy, efficiency, versatility, time, cost
Disadvantages – Survey errors and communication problems
Types of surveys
Telephone interview, Self administered survey, Personal
interview, Combination of survey methods
Purchase Intercept Technique
Can assess both shopping behavior and reason behind it
* Aids buyer recall and minimize the time lapse between purchase and
data collection
* Buyers are not the only decision makers
Executive Interviewing.
Very expensive for arranging a meeting and traveling
* Long waiting time and cancellation are common
* Require best interviewers because decision-makers can be more
knowledgeable on specific topics than interviewers
Mail intercept survey
Relatively low costs
* Sample is not representative of the general population
Omnibus Surveys
Low costs – clients share costs
* Good solution if questions are limited
* Suitable for tracking and before-after studies
Door-to-Door
Low costs – clients share costs
* Good solution if questions are limited
* Suitable for tracking and before-after studies
Pro’s and Cons of personal interviews
Pros: *
Question clarification
* Lower item nonresponse
* More complete answers
* Higher participation
* Longer interviews
* Visual aids
Cons:
Lack of respondent anonymity
* Poor for sensitive topics
* Interviewer bias
* Higher cost
Telephone survey facts:
Telephone interview became the dominant method for obtaining
information from large samples.
Pro:
Low cost
* Fast
* Absence of administrative
problems
Con:
* Type of information that
can be collected
* Screening services
* Declining response rate
* Declining number of
landlines
* Sample distortion
Self administered surveys pro’s / cons
Pros
Low cost
* Larger sample sizes
* Shorter period for collecting
data
* More representative
samples
* Survey answered at
respondents’ discretion
Cons:
* The identity of the
respondent
* Whom the respondent
consults for help in
answering questions
* The order in which the
questions are exposed and
answered
* Understanding of the
questions
what are the 4 Observational Methods
Causal Observation, Direct Observation, Physical Trace Observation, Behavior Recording Device.
Give characteristics of the 4 observations methods
Casual Observation
➢ Managers continually monitor competitive prices,
advertising activity etc. to identify problems and
opportunities.
➢ Direct Observation
Researcher observes actual behavior as it occurs. E.g. Store traffic, consumer movement form shelf to shelf
Physical Trace Measures
➢ Involves recording of natural residue of behavior
➢ E.g. The number of different fingerprints on a page was used to gauge the readership of various advertisements in
a magazine
➢ Alcohol consumption in a town without alcohol stores has been estimated from empty bottles in the garbage
Limitations of Observation Methods
Limitations - Cannot be used to observe motives, attitudes or intentions
Costly, time consuming, bias due to observant
- Qualitative Research
- Qualitative research involves collecting and analyzing nonnumerical data (e.g., text, video, or audio) to understand concepts, opinions, or
experiences
What are the types of qualitative research?
- Focus group Discussions – 8-12 participants who are led by a moderator in an in-depth discussion on one particular topic or
concept
Individual In-depth interviews –
Projective Techniques – Use when direct questions do not work i.e to extract subconscious information
Characteristics of Focus Group Discussions?
FGD is the process of obtaining possible ideas
or solutions to a marketing problem from a
small group of respondents by discussing it.
Emphasis on group dynamics, interaction, and open discussion
* Key elements are moderator, discussion guide and focus group length
- Advantages – Discussion of Complex issues, in-depth information
- Disadvantages - skilled moderator, minority viewpoints may not be heard or exaggerated
Limitations of Qualitative Methods
Potential susceptibility of the results to get misused or misinterpreted
* Results not necessarily representative of the whole population
* Moderator or interviewer’s role is extremely critical and can lead to ambiguous or misleading results
Focus group discussion pros and cons
pros:
Getting in-depth information
* Enables complex issues to be discussed
* Process highlights the differences between consumers
* Spontaneity
* Quick
cons:
Results are not directly quantifiable
Difficulty in getting attendees.
Minority viewpoints may not be heard or may be
exaggerated
* Skilled moderator
Need for Qualitative Research
- Get rough idea about things that can’t be directly observed and measured
- Information that is not possible or desirable to be obtained by using fully structured and formal methods
- The basic assumption behind qualitative methods is that an individual’s organization of relatively unstructured stimulus indicates the person’s basic
perceptions of the phenomenon and his/her reaction to it
Individual In depth Interviews
Nondirective interviews: Respondent given maximum freedom to respond
* Relaxed
* Pursuing reasons behind comments and answers
* Can be 2 hours long and tape recorded
Semi-structured or focused individual interviews: Covers a specific list of topics or sub-areas
* Specifically, effective with tech experts, business executives, thought leaders etc.
* Interviewer expertise
* Fixed time and length
* Difficulty of record keeping
Individual In depth Interviews
Nondirective interviews: Respondent given maximum freedom to respond
* Relaxed
* Pursuing reasons behind comments and answers
* Can be 2 hours long and tape recorded
Semi-structured or focused individual interviews: Covers a specific list of topics or sub-areas
* Specifically, effective with tech experts, business executives, thought leaders etc.
* Interviewer expertise
* Fixed time and length
* Difficulty of record keeping
Focus Group Key Components
Moderator
– Person hired by the client to lead the group
– Need excellent communication and organizational skills
Discussion guide
– A written outline of topics to be covered during discussion
Focus group length
– 90 minutes on average
– More questions → less time to discuss each question
Projective Techniques:
- Presentation of an ambiguous, unstructured object, activity, or person that a respondent is asked to interpret and explain.
- More ambiguous the stimulus more involvement in the task thus revealing hidden feelings and opinions
- Often used in conjunction with individual nondirective interviews
When do direct questions fail?
The issue is unimportant to respondent
-Do you prefer the packaging of Colgate or Crest? Of Kleenex or Puffs?
- Group norms, social pressures are significant
-Do you believe men have to be bread winners of their families? - Consumer doesn’t want to appear ignorant
-Do you agree or disagree with Senator Helm’s stand on aid to Russia? - Need for privacy
─ Have you been treated for mental illness?
─ Which of the following drugs have you tried? - Respondent may not be fully aware of their “true” motivation
-Why did you choose a BMW?
Commonly used techniques in IDI
- Laddering
- Step by step probing to the question that you are interested in
- Questioning progresses from product characteristics to user characteristics
Do you like this? Why? What did it do? Have you used other items like it?
Symbolic analysis
* Analyze the symbolic meaning of objects by comparing them with opposites
- Hidden issue questioning
- Focus on personal sore spots instead of general lifestyle; understand respondents’ feeling about sensitive issues in their lives; fantasies, childcare, daydreams, ideal world
When is sample appropriate?
Population size is large
- Both cost and time associated with obtaining information from the population is high
- Quick decision is needed
- To increase response quality since more time can be spent on each interview
- If census is impossible
