chapter 10 Flashcards
What is an experiment
A research approach in which one variable is manipulated and the effect on another variable is observed.
The researcher changes or manipu- lates one thing (called an experimental, treatment, independent, or explanatory vari- able) to observe the effect on something else (referred to as a dependent variable)
Causal research:
Research designed to determine whether a change in one variable likely caused an observed change in another
Experimental research is often referred to as causal (not casual) research because it is the only type of research that has the potential to demonstrate that a change in one variable causes some predictable change in another variable
to demonstrate causation (that A likely caused B), a researcher must be able to show three thing:
- concomitant variation(correlation) 2. appropriate time order of occurrence 3. elimination of other possible causal factors
Concomitant variation
A statistical relationship between two variables
To provide evidence that a change in A caused a particular change in B, one must first show that there is concomitant variation, or correlation, between A and B; in other words, A and B must vary together in some predictable fashion
There is negative and positive relations
However, concomitant variation by itself does not prove causation. Simply because two variables happen to vary together in some predictable fashion does not prove that one causes the other
Appropriate time order of occurrence
The second requirement for demonstrating that a causal relationship likely exists between two variables is showing that there is an appropriate time order of occur- rence
To demonstrate that A caused B, one must be able to show that A occurred before B occurred.
Appropriate time order of occurrence : A change in an independent variable occurring before an observed change in the depen- dent variable
Elimination of other possible causal factors
The most difficult thing to demonstrate in marketing experiments is that the change in B was not caused by some factor other than
Experimental setting
Experiments can be conducted in a laboratory or field setting.6 Most experiments in the physical sciences are conducted in a laboratory setting; many marketing experiments are field experiments
lab experiments
experiments conducted in a controlled setting. The major advantage of conducting experiments in a laboratory is the ability to control extrane- ous causal factors—temperature, light, humidity, and so on—and focus on the effect of a change in A on B. In the lab, the researcher can effectively deal with the third element of proving causation (elimination of other possible causal factors) and focus on the first two elements (concomitant variation and appropriate time order of occurrence)
Field Experiments:
Tests conducted outside the laboratory in an actual environment, such as a marketplace.
Solve problem of realism of the environment, but open up new set of problem
Research cannot control all of the spurious factors that might influence the dependent variable
Validity:
degree to which an experiment actually measures what the researcher was trying to measure
Internal validity:
extent to which the experiment is indeed measuring what it intends to measure, extent to which competing explanations for experimental results observed can be ruled out
Said to be internally valid if researcher can show that the experimental variable actually produced difference observed in the dependent variab
External validity:
extent to which causal relationships measured in an experiment can be generalized to outside persons, setting, and times
Issue: participants and setting used must be representative of others the researcher would like to project the results
Extraneous variables
history
maturation
instrument variation
selection bias
mortality
regression to the mean
testing affects
History:
intervention, between the beginning and end of the experiment, of any variable or event that might affect the value of the dependent variable
maturation
changes in participants during the course of the experiment that are a function of time (getting older more tired, etc),
Responses of people may change throughout an experiment
Problem depends on length of experiment (longer = more likely to be a problem)
instrument variation
changes in measurement instruments that might explain differences in the measurements taken
Problem in marketing experiments where people are used as interviewers/observers to measure dependent variable
selection bias
Systematic differences between the test group and the control group due to a biased selection process
Makeup of two groups may be different in projecting results when population is systematically different form rest of group
Observed different between test group and untreated control group may be due to difference between two groups
Randomization: assigning participants to test groups at random
Matching: making sure there is one-to-one match between people or other units in the test and control groups
mortality
The loss of test units or participants during the course of an experiment, which may result in non-representativeness.
No easy way to know whether lost units would have responded to treatment variable in the same way as those units that remained throughout the entire experiment
testing effects
An effect that is a by-product of the research process itself.
Main testing effects: possible effects of earlier observations on later observations.
Interactive testing effect: effect of a prior measurement on a subject’s response to a later measurement
regression to the mean
tendency of participants with extreme behaviour to move toward the average for that behaviour during the course of an experiment.
Test units may exhibit extreme behaviour because of chance, or they may have been specifically chosen because of their extreme behaviour
Controlling extraneous variables
4 basic approaches
Randomization: random assignment of participants to treatment conditions to ensure equal representation of subject characteristics.
Physical control: Holding constant the value or level of extraneous variables throughout the course of an experiment.
Design control: use of the experimental design to control extraneous causal factors.
Statistical control: Adjusting for the effects of con- founded variables by statistically adjusting the value of the depen- dent variable for each treatment condition
Experimental design:
test in which the researcher has control over and manipu- lates one or more independent variables
4 factors of experimental design
Treatment variable that is manipulated
Participants who are a part of experiment
Dependent variable that is measured
plan/procedure for dealing with extraneous casual factors
Treatment variable
The treatment variable is the independent variable that is manipulated
Manipulation refers to a process in which the researcher sets the levels of the independent variable to test a particular causal relationship, to test the relationship between price (independent variable) and sales of a product (dependent variable)
test group vs control group
A test group is a group that is exposed to manipulation (change) of the independent variable
Control group is a group in which the independent variable is not changed during the course of the experiment, it is used for comparison against the test group
Experimental notation (x)
X is used to indicate the exposure of an individual or a group to an experimental treatment
The experimental treat is the factor who effects we want to measure and compare ex: different prices, package designs, displays, etc
what is o used for
O (for observation) is used to refer to the process of taking measurements on the test units
Test units are individual groups or entities whose response to the experimental treatments is being tested ex: individual consumers, groups of consumers, retail stores, total markets or other entities that might be the targets of a firm’s marketing program
Different time periods are represented by the horizontal arrangement of the Xs and Os
Pre-experimental Designs
Studies using pre-experimental designs are difficult to interpret because such designs offer little or no control over the influence of extraneous factors
Researchers have littel control over aspects of exposure to the treatment variable (scuh as to whom and when) and measurements
Often used in commercial tes marketing, they are useful for suggesting new hypothesis but do not offer strong tests of existing hypothesis
One shot Case study design (type of pre-experimental design)
Involves exposing test units (people or test markets) to the treatment variable for some period of time and then taking a measurement of the dependent variable (X O1)
There are two weaknesses:
a) no pretest observations are made of the test units that will receive treatment and no control group → therefore extraneous activities are not dealt with and it lacks internal validity
One-group pretest- posttest design (type of pre-experimental design)
The one-group pre-test post-test design is employed most frequently for testing changes in established products or marketing strategies, the product was on the market before which is the pre-test design (O1)
The design looks like this: O1 X O2
Pre-test observations are made of a single group of participants or a single test unit (O1) that then receives the treatment. Finally, a post-test observation is made (O2). The treatment effect is estimated by O2 – O1
History is a threat: an observed change in the dependent variable might be caused by an event that took place outside the experiment between the pre-test and post-test measurements
Whereas in a lab experiment respondents can be insulated from outside influences
Maturation is another threat: observed effect might be caused by the fact that participants have grown older, smarter and more experienced
Since there is only 1 pre test observation the researcher knows nothing about the pretest trend on the dependent variable
True experimental designs
In a true experimental design the experimenter randomly assigns treatments to randomly selected test units
The random assignment is denoted by R
Randomization makes the experiment more valid
True experimental designs are better because randomization takes care of many extraneous variables, they clarify causal inference
Before and after with control group design (type of true experimental design)
The two groups of test units can be considered equivalent due to randomization, therefore they will have the same extraneous factors except for the treatment (this is why its better than pre and post test)
The true impact of the treatment variable X can only be known when the extraneous influences are removed
Two major threats to validity
a) mortality: problem if units drop out during the study and these units differ systematically form the ones that remain → this results in selection bias
b) history will be a problem if factors other than the treatment variable affect the experimental group but not the control group
After only with control group design (type of true experimental design)
The after-only with control group design differs from the one-shot case study design in the use of a control group
The test units are randomly assigned, therefore the groups should be equal
Quasi-experiments
Quasi-experiments
Used when you cant randomly assign participants to groups
When designing a true experiment the researcher usually has to make artificial environments to control independent and extraneous factors, but this leads to an issue with external validity, quasi-experimental designs deal with this issue
In quasi-experiments, the researcher lacks complete control over the scheduling of treatments or must assign respondents to treatments in a non-random fashion
Ex: a movie theatre introduces new pricing at 2 theatres, the researcher cannot randomly assign movie goers between the two theatres making this a quasi xperiment
Interrupted time series designs
Interrupted time-series designs involve repeated measurements of an effect both before and after the introduction of a treatment which interrupts previous data patterns
This is commonly used in research when using consumer purchase panels
They will take measurements of consumers purchase activity (Os), introducing a new campaign (X) and examining the panel data for an effect
Limitations of experimental research
high cists’’
- Experiments can be vert costly in both money and time
security issues
Conducting a field experiment in a test market involves exposing a marketing plan or a key element of it and competitors will know in advance what is being considered
Competitors have stolen concepts that were being tested in the marketplace and gone national before the testing company could
implementation problems
Problems that affect the implementation of the experiment include difficulty gaining cooperation with the organization, contamination problems, differences between test markets and total population and the lack of appropriate group of people or geographic area for a control group
