Experimental Design and A/B Testing Flashcards
what is a pretest?
measurements taken before delivery of the experimental stimuli
what is posttest?
measurements taken after the delivery of the experimental stimuli
what are some decisions that must be made by the researcher?
1) should there be both a pretests and posttest?
2) how many experimental groups?
3) should the experiment include a control group?
what are posttest only designs?
experiments that only include a posttests
T or F: researchers can vary the number of experimental groups
True
what is the Solomon four-group design?
an experimental design that protects the study from the biases of pretest because it uses two experimental groups and two control groups
how does the Solomon four-group design work?
- only one experimental and control group take the pretest.
- the other groups are subjected only to the intervention or the placebo and do not take the pretest.
- differences in results show if the pretest influences the results
what is the factorial design?
involves the manipulation of more than one independent variable
when are factorial designs used?
used when there is an expectation that two factors might work in concert with one another to produce an effect
Do all experiments need a control group?
not necessarily
what do control groups do?
account for confounding or disturbing variables that can affect the independent variable and the results of the dependent variable
- gives us a baseline to compare our manipulated group
what is A/B testing?
a way to compare two versions of something to figure out which performs better
- most basic kind of randomized controlled experiment
what are the applications of A/B testing?
marketing communications
web design
user experience
human factors
T or F: when conducting A/B tests, there should be multiple differences across versions A and B
False: should only be ONE differences across versions A and B
What are common A/B metrics?
clickthrough rate
time spent on page
bounce rate
