Statistics Flashcards
Cement your grasp of certain fundamental concepts in binary classification and inferential statistics: true and false positives and negatives, positive and negative predictive values, ROC curves and AUC, Bayes' Theorem, and p-values.
In an experiment with a high p-value, your data are highly ____ given a true null hypothesis.
likely
In an experiment with a low p-value, your data are highly ____ given a true null hypothesis.
unlikely
The p-value is the probability of…
obtaining an effect AT LEAST as extreme as the one observed assuming that the null hypothesis is true
A study found a difference between two means with a p-value of 0.02. Interpret this p-value in terms of many repetitions of identical studies.
If you repeated the study many times, you would find differences at least as large as observed in this study 2% of the time.
The p-value answers what question?
How likely are your data given that the null hypothesis is true?
The probability of falsely rejecting a true null hypothesis is called…
Type I error = false alarm = false positive
What two factors determine the probabilities of Type 1 and Type 2 errors?
The desired level of significance and the power of the test
The probability of falsely accepting a false null hypothesis is called…
Type II error = missed detection = false negative
Does type 1 error reject or accept the null hypothesis?
Type 1 error (odd number) rejects the null hypothesis, an “even” number
Does type 2 error reject or accept the null hypothesis?
Type 2 error (even number) accepts the null hypothesis, another even number
If you commit a type I error, what do you do to the null hypothesis?
Type I error = reject the null hypothesis even though it’s actually true = false positive
If you commit a type II error, what do you do to the null hypothesis?
Type II error = accept the null hypothesis even though it’s actually false = false negative
A false positive is also known as what kind of error?
Type I: false Positive has one vertical line
A false negative is also known as what kind of error?
Type II: false Negative has two vertical lines
What’s the formula for statistical power in terms of α and/or β?
power = 1 - β
If an experiment’s probability of type II error increases, then the statistical power ____
decreases; power = the ability to correctly reject a false null hypothesis = 1 - β
The likelihood that a study will detect an effect when there really is one to be detected is…
statistical power
A study reports no effect when in fact there was one. What kind of error is this?
Type II error = β = false negative
A study reports an effect when in fact there was no effect. What kind of error is this?
Type I error = α = false positive
What four factors affect statistical power?
- effect size
- sample size
- desired α (type I error)
- the chosen or implied β, or equivalently, the statistical power 1 – β
Given any three of these, you can find the fourth.
What are the two families of effect size indexes?
- differences between groups (risk ratio, odds ratio, Cohen’s d, Glass’s delta, etc.)
- measures of association (corr coeff r, r^2, Spearman’s rho, Cohen’s f, etc.)
T or F: the p-value is the probability of getting a false positive.
False! p = probability of seeing at least that big an effect, assuming null is true. It is actually impossible to calculate the probability that the null hypothesis is true solely from sample statistics.
In general, which is greater: the probability that the null hypothesis is true, or the p-value?
The probability that null is true tends to be greater than the p-value by a large margin.
T or F: a confidence interval is a range of values that is likely to contain an unknown population parameter.
True
