week 9 - t-tests, Comparing means Flashcards
hypothesis testing (formal definition of p value)
p value = P(D given not-H)
OR
p value = P(D given H-null)
D = “our observed data or more extreme data”
Low p value?
- We have observed something that would be VERY
IMPROBABLE if the null hypothesis true - so we decrease believe in null Hypothesis
- (and increase belief in alternative)
null hypothesis
no change (i.e., H = our drug enhances IQ, null-H = the drug doesn't change IQ)
alpha
Compute alpha (α): α = 1 - (confidence level / 100)
an acceptable significance value
comparing means
the population model parameter of interest is the difference between the two means:
μ1 - μ2
We are working with means and estimating the standard error of their difference using the data –>
SO the sampling model is a Student’s t.
z or t?
If you know σ, use z. (That’s rare!)
Whenever you s to estimate σ,
use t.
The confidence interval we build is called a two-sample t-interval (for the difference
in means).
The corresponding hypothesis test is called a two-sample t-test.
The interval looks just like all the others we’ve seen—the statistic plus or minus an estimated margin
of error: ȳ1 - ȳ2 +/- ME
REMEMBER: if you know the standard deviation of the population then use the ……… distribution
the Normal distribution.
If you do not know the standard deviation of the population then you MUST use the Student-t distribution.
Hypothesis Testing
Hypothesis testing is the use of statistics to determine the probability that a given hypothesis is true. The usual process of hypothesis testing consists of four steps.
- Formulate the null hypothesis H_0 (commonly, that the observations are the result of pure chance) and the alternative hypothesis H_a (commonly, that the observations show a real effect combined with a component of chance variation).
- Identify a test statistic that can be used to assess the truth of the null hypothesis.
- Compute the P-value, which is the probability that a test statistic at least as significant as the one observed would be obtained assuming that the null hypothesis were true. The smaller the P-value, the stronger the evidence against the null hypothesis.
- Compare the p-value to an acceptable significance value alpha (sometimes called an alpha value). If p
Hypothesis testing, alpha value
- P(concluding H-0 is false given that H-0 is true)
- the probabilitiy that we will conclude we have a real finding when we actually don’t
- False positive rate
- Type I error rate
What can we conclude if the p value is large?
– Unfortunately, not much.
● If p is large, it might be that H0 is true…
● … but it might also be that:
– Our sample is too small
– The population is too varied
– The real effect in the population is too small
2-sample t-test
difference between the means of two samples
df = n1 + n2 -2
Power
– P(concluding that H0 is false | H0 is false)
– The probability that we will conclude we have a real
finding, when we really do.
– True positive rate
margin of error
the amount of random sampling error in a survey’s results
The margin of error is usually defined as the “radius” (or half the width) of a confidence interval for a particular statistic from a survey.
In a confidence interval, the range of values above and below the sample statistic is called the margin of error.
We could devise a sample design to ensure that our sample estimate will not differ from the true population value by more than, say, 5 percent (the margin of error) 90 percent of the time (the confidence level).
The margin of error can be defined by either of the following equations.
ME = Critical value x Standard deviation of the statistic
ME = Critical value x Standard error of the statistic
standard error
A Bayesian interpretation of the standard error is that although we do not know the “true” percentage, it is highly likely to be located within two standard errors of the estimated percentage. The standard error can be used to create a confidence interval within which the “true” percentage should be to a certain level of confidence.
