Reading Quiz 11 Flashcards
steps/template used to conduct a significance test (aka hypothesis test)
- hypotheses
- conditions
- calculations
- interpretations
hypotheses step
identify population of interest and parameter you want to draw conclusions about
state null hypothesis (Ho), alternate hypothesis (Ha), and significance level (α)
null hypothesis (Ho)
statement being tested in a significance test
significance test designed to assess strength of the evidence against the null hypothesis
null hypothesis usually statement of
no effect
no difference
no change from historical values
alternative hypothesis
the claim about the population that we are trying to find evidence for
significance level
denoted by α
represents the cutoff between rejecting and failing to reject the null hypothesis
most of time the specified, if not choose one
common = 1% and 5%
conditions step
state the appropriate inference procedure
verify the conditions for using it
SRS, Normality, independence
SRS condition
data is obtained from an SRS of the population of interest
Normality condition (for means)
population distribution is Normal, large sample size (30+) to apply central limit theorem, or observe and sketch the shape of the histogram (it’s enough if the distribution is roughly symmetric and unimodal)
independence condition
assume individual observations are independent
when sampling without replacement must verify that population size N is at least 10 times the sample size
calculations step
calculate test statistic and find p value of test statistic
test statistic general form
z = (estimate - hypothesized value)/(standard deviation of the estimate)
test statistic for one sample z test for means
z = (x̅-μo)/(σ/√n) μo= mu not
p value of a significance test
the probability, computed assuming that Ho is true, that the test statistic would take a value as extreme or more extreme than that actually observed
interpretations step
interpret the results of the test (fail to reject or reject the null hypothesis)
statistically significant
if the p value is less than or less than or equal to the significance level α, our data is statistically significant at this level and we reject the null hypothesis
if the p value is greater than the significance level α we fail to reject the null hypothesis
small p values are evidence…
large p values fail to give evidence…
against Ho because they say that the observed result is unlikely to occur when Ho is true
against Ho
tests that can also be performed by using a confidence interval
two tailed significance tests
Ho true and reject Ho
type I error (α)
Ho true and fail to reject Ho
correct decision
Ho false and reject Ho
correct decision aka power
Ho false and fail to reject Ho
type II error (β)
important information about type I and type II errors
if a significance test has a fixed significance level α then α is the probability of making a type I error
the probability of a type II error is denoted by the symbol β
there is a(n) … relationship between type I and type II errors
inverse
power of a significance test
the probability that a fixed level α significance test will reject Ho, the null hypothesis, when a particular alternative of the parameter is true is called the power of that test against that alternative
power of a significance test easier definition
the probability that the null hypothesis will be rejected if it is false
1 - P(type II error) = 1 - β
two main ways to increase the power of a significance test
- increase the sample size
- increase the significance level
there are other ways to do so too but these are the ways that are generally within the control of the experimenter
increasing the significance level…
increases the probability of making a type I error, which decreases the probability of making a type II error, which makes 1 - β a larger quantity
the higher alpha, the lower beta, and vice versa
- True or False: The significance test is designed to assess the strength of the evidence against the null hypothesis.
true
In doing statistical tests, the first step is to identify what you want to make conclusions about. Are you making conclusions about sample statistics or population parameters? Is that always the case or does it depend upon the problem?
Population parameters. Always.
What does a “null hypothesis” typically state?
That there is no difference between two parameters, or no effect, or no change (or that a parameter is equal to a certain value).
A significance test works by assessing how likely the ______ _____ would be if the ____ _____ were true.
observed outcome, null hypothesis
True or False: the p-value is the probability of getting exactly the results we observed, presuming the null hypothesis to be true.
False. The probability of getting exactly the results we obtained is almost always very small (or even theoretically 0 when dealing with continuous distributions). The p value is the probability of getting results as extreme, or more extreme, than the actually observed results; “extreme” means “far from what we would expect if the null hypothesis were true.”
We are more likely to reject the null hypothesis of “no difference” or “no effect,” and infer that there is a difference or an effect, when the p-value is large, or small?
small
The null hypothesis has to do with a population parameter; in analyzing your sample data you calculate a ______ that estimates that population parameter.
Statistic (the phrase, “sample statistic” is correct but redundant.)
When a drug company researcher is hoping to find evidence that a drug is better than placebo, is the researcher wishing to reject, or fail to reject, the null hypothesis?
Reject. The null hypothesis would be that drug and placebo are equal in effect.
What is the meaning of the significance level, or alpha?
It’s a threshold level for the p-value that we consider decisive, with which the obtained p-value is compared. We then decide whether to reject or fail to reject the null hypothesis.
Do we reject the null hypothesis when the p value is less than alpha, or greater than alpha?
less than alpha
If a test is statistically significant at the .05 level, what does that mean?
That the p-value obtained is less than or equal to .05.
Someone finishes writing up a statistical test by saying, “In conclusion, p = .021.” What step of the
“inference toolbox” are they leaving out, that should come after what they said?
Interpreting the results in the context of the problem. So they should say something like, “Therefore we reject the hypothesis that drug and placebo are equal; our study gives evidence that our drug is more effective than the placebo.”
When we are testing the hypothesis that a population mean is equal to a certain hypothesized value, in
the unlikely situation where we know the population standard deviation, what is our test statistic formula?
z = (xbar - mu not)/(sigma/sqrt(n))
What distribution does the one-sample z statistic, a.k.a. the standardized sample mean, have when the null hypothesis is true?
the standard normal distribution
Please explain why the two-sided p-value is double that of the one-sided p-value.
The p-value is the probability of getting results as extreme as, or more extreme than, the results obtained. For a two sided test, we add the probability of getting results extreme in both directions to get the total p value; for symmetrical distributions, those two probabilities are equal, thus amounting to twice the value for any single direction.
True or False: If you obtained a 95% confidence interval for a mean that ranged from 10 to 30, then a
null hypothesis that the mean was equal to any value outside that range would be rejected and a null hypothesis of a mean within that range would not be rejected, at the .05 level, using a two-sided test.
true
True or False: If you report the p-value itself, rather than saying, that p
true
True or False: if the significance level is set at .05 then a p-value slightly less than .05 is technically considered statistically significant but should not be considered practically significant.
true
If there are bad design problems, is it likely, or unlikely, that sophisticated inferential statistical analysis techniques can get around these problems to produce valid inferences?
unlikely
Suppose the null hypothesis is that a drug has no effect. If this is true and our analysis yields the decision that the drug is effective, that is a _____ error; if the drug has an effect but our analysis concludes that there is no effect, that is a ______ error.
type I, type II
If we set alpha at .01, what is the probability of a type 1 error given that the null hypothesis is true?
The probability is .01.
The probability that a fixed level alpha significance test will reject the null when a particular alternative value of the parameter is true is called the _____ of the test against that alternative.
power
The power of a test is what function of the probability of a type 2 error?
1 minus probability of type 2 error.
True or False: the p-value tells what would happen if we tested many samples, when the null hypothesis is true; the power tells what would happen if we tested many samples, when a particular alternative hypothesis is true.
true
What happens to power as you increase sample size, all other things equal?
it increases
What happens to power as you increase the significance levels, all other things equal?
it increases
when inside confidence interval vs when outside confidence interval
statistically significant = outside confidence interval
not statistically significant = inside confidence interval
p value second
the likelihood of getting similar results assuming Ho is true
p less than or equal to alpha
statistically significant
reject
p greater than or equal to alpha
not statistically significant
fail to reject
ART BFF
Alpha (type I), Reject Ho, actually True
Beta (type II), Fail to reject Ho, actually False
