CH11 Experimentation & Test Markets

Question 1

define an experiment

Answer 1

experiment: a research approach in which one variable is manipulated and the effect on another variable is observed

Question 2

what is the purpose of experimental research?

Answer 2

experimental research is used to test whether a change in an independent variable causes a predictable change in a dependent variable

Question 3

what is causal research?

Answer 3

causal research: research designed to determine whether a change in one variable likely caused an observed change in another

Question 4

what are the 3 criteria to establish causality in experimental research?

Answer 4

1. correlation or concomitant variation
2. appropriate time order of occurrence
3. elimination of other possible causal factors

Question 5

what are the 2 types of experimental settings?

Answer 5

1. laboratory experiments
2. field experiments

Question 6

what is the difference between laboratory experiments and field experiments?

Answer 6

laboratory experiments are conducted in controlled settings, while field experiments take place in the actual marketplace

Question 7

what are internal and external validities in experimentation?

Answer 7

internal validity refers to ruling out competing explanations of experimental results, while external validity relates to generalizing the findings to other settings

Question 8

what are extraneous variables in experimental research?

Answer 8

extraneous variables: other independent variables that may affect the dependent variable and hinder the ability to conclude a causal relationship between the treatment variable and the dependent variable

Question 9

list some examples of extraneous variables

Answer 9

1. history: intervention, between the beginning and end of an experiment, of outside variables or events that might change the dependent variable
2. maturation: changes in subjects occurring during the experiment that are not related to the experiment but that may affect subjects’ response to the treatment factor
3. instrument variation: changes in measurement instruments (eg. interviewers or observers) that might affect measurements
4. selection bias: systematic differences between the test group and the control group due to a biased selection process
5. mortality: loss of test units or subjects during the course of an experiment, which may result in a nonrepresentativeness
6. testing effect: the effect that is a by-product of the research process itself
7. regression to the mean: the tendency of subjects with extreme behaviour to move toward the average for that behaviour during the course of an experiment

Question 10

what are the 4 basic approaches used to control extraneous factors in experiments?

Answer 10

1. randomization: the random assignment of subjects to treatment conditions to ensure equal representation of subject characteristics
2. physical control: holding constant the value or level of extraneous variables throughout the course of an experiment
3. design control: use of the experimental design to control extraneous causal factors
4. statistical control: adjusting for the effects of cofounded variables by statistically adjusting the value of the dependent variable for each treatment condition

Question 11

define experimental design

Answer 11

experimental design: the test in which the researcher has control over and manipulates one or more independent variables

Question 12

what are the 4 elements of an experimental design?

Answer 12

1. a treatment variable
2. subjects
3. a measured dependent variable
4. a plan for dealing with extraneous causal factors

Question 13

what are some limitations of experimental research?

Answer 13

• high cost
• security issues
• implementation problems

Question 14

what are the characteristics of pre-experimental designs?

Answer 14

pre-experimental designs offer limited control over extraneous factors and include designs such as the one-shot case study and the one-group pretest-posttest design

Question 15

provide an example of a pre-experimental design

Answer 15

an example of a pre-experimental design is the one-shot case study design, which involves exposing test units to a treatment variable and measuring the dependent variable afterward, without a control group or pretest observations

Question 16

describe the one-shot case study design

Answer 16

the one-shot case study design is a pre-experimental design where test units are exposed to a treatment variable, and the dependent variable is measured afterward, without a control group or pretest observations

Question 17

what are the weaknesses of the one-shot case study design?

Answer 17

its weaknesses include the absence of pretest observations and a control group, making it difficult to establish causal inferences and lacking internal and external validity

Question 18

what is the one-group pretest-posttest design?

Answer 18

the one-group pretest-posttest design is a pre-experimental design where a single group of subjects or test units is measured before and after receiving the treatment, without a control group

Question 19

what are the strengths and weaknesses of the one-group pretest-posttest design

Answer 19

its strengths include simplicity and cost-effectiveness

however, it is vulnerable to threats like history and maturation, and without a control group, it cannot estimate the true impact of the treatment

Question 20

what are true experimental designs and how do they differ from pre-experimental designs?

Answer 20

true experimental designs involve the random assignment of test units to experimental and control groups, providing better control over extraneous factors and enabling causal inferences

they differ from pre-experimental designs by incorporating essential elements such as randomization, control groups, and pretest observations

Question 21

explain the before and after with control groups. how is the treatment effect estimated in this design?

Answer 21

the before and after with control group design is a true experimental design where test units are randomly assigned to experimental and control groups. pretest and posttest measurements are taken for both groups.

the treatment effect is estimated by comparing the difference between the pre- and postmeasurements of the experimental group with the difference in the control group, removing the influence of extraneous factors.

Question 22

what are the major threats to validity in the before and after with control group design?

Answer 22

the major threats to validity in the before and after with control group design are mortality (when units drop out and differ systematically) and history (when events outside the experiment influence the experimental group differently)

these threats can affect the internal validity of the design

Question 23

describe the after-only with control group design. what are its limitations?

Answer 23

the after-only with control group design is a true experimental design where test units are randomly assigned to experimental and control groups, but only postmeasurements are taken

its limitations include the lack of premeasurements, making it difficult to establish the baseline and estimate the true treatment effect accurately

Question 24

what are quasi-experiments, and why are they used in marketing research?

Answer 24

quasi-experiments are studies in which the researcher lacks complete control over treatment scheduling or must assign respondents to treatments in a nonrandom manner

they are used in marketing research due to cost and field constraints that make it challenging to exert direct control or randomization. quasi-experiments provide more feasible options in field settings

Question 25

explain the interrupted time-series design. how does it differ from the one-group pretest-posttest design?

Answer 25

the interrupted time-series design involves repeated measurements of an effect both before and after a treatment is introduced, “interrupting” previous data patterns

it differs from the one-group pretest-posttest design by including multiple pre- and postmeasurements, allowing for better understanding of the effects of extraneous variables and capturing trends in the dependent variable over time

Question 26

what are the weaknesses of the interrupted time-series design?

Answer 26

the weaknesses of the interrupted time-series design include the potential influence of confounding variables and threats to internal validity such as history, maturation, and regression to the mean. the design also assumes that the treatment effect is immediate and does not consider potential delayed effects

Question 27

how does the multiple time-series design address the limitations of the interrupted time-series design?

Answer 27

the multiple time-series design addresses the limitations of the interrupted time-series design by including a control group or multiple comparison series that are not exposed to the treatment

by comparing the treatment group with the control group(s), it helps control for confounding variables and provides a better estimate of the treatment effect

Question 28

what is the importance of selecting an appropriate control group in the multiple time-series design?

Answer 28

selecting an appropriate control group in the multiple time-series design is crucial to ensure valid comparisons between the treatment and control groups

the control group should be similar to the treatment group in all relevant aspects except for the exposure to the treatment. this helps to isolate and attribute the observed effects to the treatment rather than other factors

Question 29

what is test marketing?

Answer 29

test marketing involves testing a new product or marketing mix element using experimental or quasi-experimental designs

Question 30

define the 4 different types of test markets

Answer 30

1. traditional test markets: choosing actual markets for introduction of test products and track restuls
2. simulated test markets (STMs): simulations of traditional test markets
3. virtual test markets (VTMs): putting consumers into something like a realistic shopping environment on a computer
4. controlled test markets: tests conducted in markets where research firms have panels of consumers they can track regarding their purchase behaviour

Question 31

list the 2 key benefits of conducting test marketing

Answer 31

1. the test market provides a means to obtain accurate sales potential estimates for new products or strategies in realistic or semi-realistic market conditions; these estimates can be used to predict future financial performance
2. test markets help identify weaknesses in products or marketing strategies, allowing management to address these issues before national distribution, which is more costly and challenging.

Question 32

what are the 4 major factors that should be taken into account in determining whether to conduct a test market?

Answer 32

1. weigh the cost and risk of failure against the probability of success and associated profits
2. consider the likelihood and speed with which competitors can copy your product and introduce it on a national basis
3. consider the investment required to produce the product for a traditional test market versus the investment required to produce the product in the quantities necessary for a national rollout
4. consider how much damage an unsuccessful new product launch would inflict on the firm’s reputation

Question 33

what are the steps involved in conducting a test market study?

Answer 33

1. defining objectives
2. selecting an approach
3. developing procedures
4. selecting markets
5. executing the plan
6. analyzing results