“Conducting Experiments for Gaining Customer Insights aka data marketing experiments Flashcards
casual conclusion
how one factor caused a change in another variable of interest
casual research
aka experiments
allows a firm to make cause and effect statement
cause and effect stamenent
can be made if it can be shown with a high level of internal validity that the casual variable or indepedent variable was responsible for the change in the outcome variable or dependent variable
objective of experimental design
to create groups of respondents in a way that observed changes in one group can only be attributed to the casual variable
done through experimental design
The main goal of experimental design is to figure out if one thing (the cause) is really affecting something else (the effect). To do this, we create different groups of people (respondents) and treat them differently in a way that we can clearly see if the changes in one group are caused by the factor we’re testing
experimental design
carefully gather data and then evaluates it to draw conclusions then do statistical analysis
when looking at causality, use a control and trewtaenet group
in the treatment group you vary the casual factor aka the indepedent factor
then observe the level of dependent variable
three conditions allow us to infer casualty
- if the two groups are equivalent in all other respects minus what u are testing
- of there is a stisatical association
- we can rule out other possible explanations for the observed association
if yes we can conclude that the independent variable causes a change in the dependent variable
customer focused experiments should balance two types of validity
internal validity and external
internal validity
an experiments ability to unambiguously show a cause and effect relationship –> the extent to which we can attribute the change in dependent variable to the indepdent variable and not to other factor
major concern for internal validity
have to ensure that the treatment group and control group are equal
achieved through random assignment or matched samples
random assignment
occurs when each experimental unit (ex: each salesperson) is randomly assigned for inclusion in the control and treatment group
each salesperson has an equally likely chance of being assigned to either the treatment or control group
statistically compare each group
use this but if not possible, use matched samples
if treatment and control are unequal
we have selection bias – when the treatment group is chosen using a methodology that precludes random assignment
random assignment is not always feasible however
matched sample
as a technique for generating a control group is used when legal, ethical and practical consideration preclude random assignment
external validity
the extent to which results of the experiment can be generalized or extrasploted to oder people, settings and time
do the results from one experiment with high internal validity apply to other situations
differences in setting can limit external validity
systemTIC BAISES
can threaten the random assignment protocol
threats to internal validity
- history–> any variable or events other than the ones in the xperiemnet that affect the value of the Fdepdenet variable
Events or variables outside the experiment that affect the dependent variable during the course of the experiment.
Example: If you’re testing the effect of a new advertisement on sales, but during the experiment, a major economic event (like a sudden price increase for competitors) occurs, this could influence the sales, making it hard to tell if the ad itself caused the sales changes
- maturation–> any economic, biological or psychological processes that systematical vary with passage of time
Changes within participants over time that are not due to the experimental treatment but to natural processes (e.g., aging, learning, fatigue).
Example: In a long-term study on the impact of a new training program, employees might improve over time due to gaining experience or becoming more familiar with their tasks, not because of the training itself
- instrument varaiton–> a change in the measuring instrument t
- experimental unti moratloty–> differential loss of respondents from dif groups
- pre messuremnet/ interactive testing-
- selection bias
- statically regression –> special type of selection bias
- enchahing effect of selection bias –> non random assignment
external validity threats
- surrogate situation–> when the expeiemtal task is so contrived that it is unlikely to be a reasonable proxy of the reality that the researcher wants to study
- measurement of dependent variable: whe deponent variable does not map on to the behaviour or construct of interest
- meaduremtn timing- If the timing of measurements in the study is not optimal or does not align with when the effects would occur in the real world, the findings might lack relevance when applied outside the study.
- sample selection –> sample may not be representative of the population
internal vs external. which is more desirable?
internal is necessary for drawing casual conclusions so even if it has high external not good
customer focused firms should strive to optimize both
use high internal to draw conclusions but use external to go beyond the conlsisions
there is a trade off between external ad internal for single customer focused research studies
pseudo (bad) designs for customer focused expriemets
many lack internal validity –. cant establish causality
1. one shot cause study –> lacks a control group that is equivalent to the treatment
2. one group pre/ post test design–> faulty pre post
3. static group comparison
- show that there is a need for a control group that is the same as the tatrement and best accomplished through rands assignment
true (good) designs for customer focused exorieennts
- post test only, control group design–> rather than creating the tatrement group based on judgement or cmveneimce, experimental units are randomly assigned to the treatment or control group
- pre test/ post test control group design–> before, exposing he treatment group to the treatment, but after random asisgmemt, a pre measurement is taken on the deponent variable. serves as the baseline from which changes in the deponent variable can be evaluated
- avdnavced casual experiments
- time series or quasi experiments
overall
- establish two equivalent groups aka control and treatment , established through random assignment or if need be matching
- researcher has to carefully evaluate the study and its context to asses threats to external and indternal. internal - high, casual inference can be made. external = low, care should be taken o project the results beyond the context of the current experiment
- conducting the approbate statistical analysis should answer 3 questions: iOS the difference statistically sigfniacant, how large is the impact, os the impact pos or neg. we use ANOVA
- researcher should assess the applicably of the results. just bc results are statistically significant may not mean they are relevant, useful of applicable
three types of marketing research
exploratory, descriptive, casual
(eiley digs Chloe)
exploratory
methods: case studies, focus groups
felxible, versatile , discobery of ideas and insights
descriptive
surveys, panels
preplanned, structured, conclusive
describe market characteristics or functions