Central Limit Theorem (CLT) , Confidence Intervals, Hypothesis Testing and A/B Testing Flashcards
To reinforce key concepts in inferential statistics, including the Central Limit Theorem, confidence intervals, statistical significance, hypothesis testing, and A/B testing. These flashcards will help learners build a strong foundation in statistical analysis, improve their ability to interpret results, and apply statistical reasoning to real-world data problems
What does the Central Limit Theorem (CLT) state?
The CLT states that the sampling distribution of the sample mean approaches a normal distribution as the sample size increases, regardless of the population’s distribution.
This is crucial in inferential statistics, as it allows normal-based methods to be applied to non-normal populations.
What is the minimum sample size recommended for the CLT to hold?
Typically, 30 or more is considered sufficient.
A larger sample size makes the approximation to normality stronger.
True or False:
CLT applies only to normal distributions.
FALSE
CLT applies to any population distribution as long as the sample size is large enough.
How does sample size affect the standard error?
A larger sample size reduces the standard error.
The standard error is given by σ/√n, so increasing n decreases variability.
Fill in the blank:
The mean of the sampling distribution is equal to the _____.
Population mean (μ)
The sample mean is an unbiased estimator of the population mean.
What is a confidence interval?
A range of values that likely contains the true population parameter with a specified probability.
Confidence intervals provide a measure of uncertainty in estimation.
What is the most commonly used confidence level?
95%
Other common levels are 90% and 99%, depending on the application.
How do you interpret a 95% confidence interval?
We are 95% confident that the true population parameter lies within this interval.
It does not mean that there is a 95% chance the parameter is in the interval.
How does increasing the confidence level affect the confidence interval?
It widens the confidence interval.
Higher confidence means greater uncertainty, requiring a broader range.
How does sample size impact confidence intervals?
Larger sample sizes narrow the confidence interval.
More data reduces uncertainty, leading to a more precise estimate.
What is a p-value?
The probability of observing the data if the null hypothesis is true.
A lower p-value suggests stronger evidence against the null hypothesis.
What is the common threshold for statistical significance?
0.05 (5%)
If p<0.05p < 0.05p<0.05, we reject the null hypothesis in favor of the alternative hypothesis.
True or False:
A p-value of 0.03 proves that the null hypothesis is false.
FALSE
A small p-value suggests evidence against the null but does not prove it is false.
What is a Type I error?
Rejecting the null hypothesis when it is actually true.
Also known as a false positive error.
What is a Type II error?
Failing to reject the null hypothesis when it is actually false.
Also known as a false negative error.
How is the significance level (α) related to Type I error?
αis the probability of making a Type I error.
A lower α reduces Type I error but increases Type II error.
What is a one-tailed hypothesis test?
A test where the alternative hypothesis specifies a directional effect.
Example: Testing if a new drug increases recovery time.
What is a two-tailed hypothesis test?
A test where the alternative hypothesis does not specify a direction.
Example: Testing if a new drug has any effect (increase or decrease).
Which statistical test is used for comparing means between two independent groups?
t-test
A two-sample t-test is used when comparing two group means.
Which test should you use when population variance is unknown?
t-test
The t-test is used instead of a Z-test when variance is unknown.
What is the null hypothesis in an A/B test?
That there is no difference between the two groups being tested.
The goal of A/B testing is to determine if the new variant outperforms the control.
What is the alternative hypothesis in an A/B test?
That there is a statistically significant difference between groups.
A/B tests often test improvements in conversion rates, click-through rates, etc.
True or False:
A large sample size reduces the likelihood of Type I errors.
FALSE
A large sample size improves power but does not directly reduce Type I errors.
Why do we use randomization in A/B testing?
To ensure no systematic bias between groups.
Random assignment increases the validity of results.
What is statistical power?
The probability of correctly rejecting the null hypothesis when it is false.
A higher power means a lower risk of Type II error.
What sample size is typically recommended for an A/B test?
Large enough to detect a meaningful difference.
Power analysis is used to determine the required sample size.
How can you reduce Type I error?
Lower the α level (e.g., from 0.05 to 0.01).
However, this increases the chance of Type II error.
What is the Bonferroni correction used for?
To adjust for multiple comparisons and reduce Type I error.
It divides α by the number of tests performed.
What does the effect size measure in A/B testing?
The magnitude of the difference between groups.
Cohen’s d is a common measure of effect size.
