A/B Testing

Question 1

What variables are used

Answer 1

Binary (yes/no), continuous variables (duration)

Question 2

Fisher’s Exact Test

Answer 2

Binary variable, small sample size (34). Random sample, independent observations.

Question 3

Pearson’s Chi-Square Test

Answer 3

Binary variable, large sample size (214). Random sample, independence, cell frequency of 5 or more.

Question 4

Student’s t-test, F-test

Answer 4

Continuous variable, large sample size (or small sample size (31), but normal distribution) variance unknown, similar/equal variance (F_critical = 1.525, so if F<F_critical, equal variances)

Question 5

Normal distribution can be tested by …

Answer 5

Histogram, box plot

Question 6

Equal variance can be tested by …

Answer 6

F-test, Levene’s test

Question 7

Welch’s t-test

Answer 7

Continuous variable, large sample size (or small sample size (38), but normal distribution) variance unknown, NOT similar/equal variance

Question 8

Mann-Whitney U test

Answer 8

Continuous variable, small sample size, NOT normal distribution