Survey errors +++

Question 1

Respondents thought they knew the answer but they really didn’t.

Answer 1

Response bias

Question 2

We got the results from our phone survey. We got a lot more females in their 30s and 40s than are in the actual population.

Answer 2

Non-response bias

Question 3

The Coke example/ the language online example –> loved the website but people didn’t want to learn languages online. Researched the wrong thing/ asked the wrong questions

Answer 3

Surrogate error

Question 4

A survey asks “how much would u pay for this product” but also didn’t leave enough space for people to answer

Answer 4

Instrument error

Question 5

I bought a list of names of software engineers to send my survey out in the Silicon Valley area, but a lot of emails on the list were bad because many of the engineers had moved to other companies.

Answer 5

Frame error

Question 6

Hired someone with poor computer skills who did a bad job entering the data (made mistakes typing)

Answer 6

Processing (input) error

Question 7

Polled the wealthiest in the area to determine the demand for weekly cleaning. Turned out they all had a full time maid. Didn’t define my population well.

Answer 7

Population error

Question 8

The person I hired to conduct the survey was rude and in a hurry.

Answer 8

Interviewer error

Question 9

Even though I told the staff in the company to get an even mix of ages and genders to complete the survey they ended up getting more old women cause that was the easiest.

Answer 9

Selection error

Question 10

Sample design errors

Answer 10

• selection error
• frame error
• population error

Question 11

Measurement error

Answer 11

• processing (input) error
• response bias
• non-response bias
• interviewer bias/error
• instrument bias/error
• surrogate information error

Question 12

A market research experiment:

Answer 12

The researcher changes an explanatory, independent marketing variable (e.g. the marketing mix) to observe changes in the dependent variable.

Question 13

Causal research

Answer 13

The only type of research that can demonstrate that a change in one variable causes a predictable change in another variable

Question 14

Laboratory experiments

Answer 14

Conducted in a controlled setting

Question 15

Lab experiments- advantages

Answer 15

• Ability to control all variables
• Greater internal validity

Question 16

Lab experiments- disadvantages

Answer 16

• May not have good external validity
• Not transferrable to the actual marketplace

Question 17

Field experiments

Answer 17

Tests conducted outside the laboratory

Question 18

Field experiments- advantages

Answer 18

• Creates realism of the environment

Question 19

Field experiments- disadvantages

Answer 19

• Internal validity
• No control over extraneous factors
  – Weather, temperature, humidity
  – Light, power, Internet speed
  – People present, influences on test subject
  – Access to information

Question 20

Validity

Answer 20

Degree to which an experiment actually measures
what it is trying to measure

Question 21

Internal validity

Answer 21

The extent to which competing explanations can be ruled out.

Question 22

External validity

Answer 22

The extent to which causal relationships can be generalized to outside persons, settings, and times

Question 23

Lab experiments

Answer 23

• High internal validity
• Low external validity

Will the results be the same in the real world though?

Question 24

Field experiments

Answer 24

• Low internal validity
• High external validity

Question 25

Experimental design

Answer 25

• control group (experimental vs control group) –> random selection
• A/B test for completely new product

Question 26

Extraneous variables- threats to experimental validity

Answer 26

– History- intervention between the beginning and end of an experiments
– Maturation- changes in subjects occurring during the experiment
– Instrument Variation- changes in measurement instruments
– Selection Bias- systematic differences between the test group and the control group
* Randomization or matching
– Mortality- loss of test units or subjects during the course of an experiment
– Testing Effect - effect that is a by-product of the research process itself (prior exposure, e.g., to product, ad, etc.) – Regression to the Mean (see Hawthorne effect*)

Question 27

Observation research definition

Answer 27

Systematic recording of behaviors, mostly without communicating with people involved.

Question 28

Approaches to observation research:

Answer 28

– Natural Versus Contrived Situations
– Open Versus Disguised Observation
– Structured Versus Unstructured
– Human Versus Machine Observers
– Direct Versus Indirect

Question 29

Advantages of observations over surveys:

Answer 29

– See what people actually do + physical reactions
– Avoids survey bias (e.g., interviewer, response)
– Quick, accurate data collection (with technology)

Question 30

Disadvantages of observation vs surveys

Answer 30

– Researcher does not learn motives
– Cannot determine future behavior
– Time-consuming and expensive (by humans)

Question 31

Four variations of a mystery shopper:

Answer 31

– Level 1 - a simple phone call for basic info
– Level 2 - a quick purchase, little or no interaction
– Level3-scripted conversation: assess basic knowledge, product options
– Level4– in-depth assessment of technical knowledge of product solution

Question 32

Dependent vs independent variables (experimental vs treatment)

Answer 32

Treatment variable: Independent variable that is manipulated in an experiment

Experimental variable: The dependent variable that is measured

Question 33

Four elements included in an experimental design:

Answer 33

1. The treatment, or experimental, variable (independent variable) that is manipulated
2. The subjects who participate in the experiment
3. Adependent variable that ismeasured
4. Some plan or procedure for dealing with extraneous causal factors

Question 34

Traditional marketing vs big data

Answer 34

Traditional:
Traditional marketing research wants to learn about how people feel about a product, a company, or an ad. Researchers want to better understand what people were thinking when they made or didn’t make a purchase decision. Marketing researchers hope that by having a better understanding of a person’s feelings and how they think, then this will lead to a better comprehension of “why.”

Big data:
Big data and marketing analytics take a different tack. Ifthese new tools can accu- rately predict what people will do, then “why” isn’t really important. As you can imagine, this has led to conflict in some organizations between traditional marketing researchers and persons working in marketing analytics using big data.

Question 35

Discussion guide

Answer 35

A discussion guide is a written outline of the topics to be covered during the session. Usually, the moderator gener- ates the guide based on research objectives and client information needs.

Question 36

Projective tests

Answer 36

Technique for tapping respondents’ deepest feelings by having them project those feelings into an unstructured situation.

The rationale behind projective tests comes from the knowledge that people are often reluctant or unable to reveal their deepest feelings. In some instances, they are unaware of those feelings because of psychological defense mechanisms.

• word association test
• analogies
• sentence and story completion tests
• cartoon tests
• story telling
• third person technique