Hypothesis Testing Flashcards
is it true that random samples don’t led to any sampling error
NO - they DO have sampling errors
why do random samples have sampling error
because we are taking a SAMPLE of the population to study NOT the entire population so there has to be chance differences between data
what does hypothesis testing compare
data to what we would EXPECT to see if a null hypothesis was TRUE (don’t reject)
when is a null hypothesis rejected
if the data is TOO UNUSUAL compared to what we expected from if the null hypothesis was TRUE
what phrase is associated with parameter estimation
how large the effect is and what is the uncertainty
what phrase is associated with hypothesis testing
whether there IS an effect
why are null hypothesis made
for the purpose of ARGUMENT
what view embodies null hypothesis
skeptical point of view
what is the null hypothesis FREQUENTLY based as
the population parameter has ZERO interest (ie. there is no effect, no correlation or no difference)
what happened to reject the null hypothesis (what kind of result)
an interesting result (ie. there was an effect, was a difference, was a correlation)
what hypothesis is paired with null hypothesis
alternative hypothesis
what does the alternative hypothesis represent
ALL the other parameters EXCEPT those stated by the null hypothesis (ie. there is an effect due to this parameter)
what hypothesis is tested with the data
NULL hypothesis
what hypothesis is not tested directly
the alternative hypothesis
when do we FAIL to reject a null hypothesis
when data is CONSISTENT with the expected data in null hypothesis
do we accept null hypotheses
NO
when do we reject null hypotheses
when data is NOT consistent with expected results in null hypothesis
if we reject the null hypothesis what is accepted
the alternative hypothesis
how does rejecting the null hypothesis help us identify the population parameter
helps identify which direction (according to alternative hypothesis) the true value of the parameter is
what steps are done in the hypothesis testing
- state the hypotheses (both alternative and null)
- determine the test statistic
- determine null distribution
- quantifying uncertainty (based on p value)
- determining statistical significance
what does a two sided test mean
an alternative hypothesis has two outcomes to reject the null hypothesis
(ie. if null says expectation is for data to be equal, alternative can reject it by having data greater than or less than that value)
what is the test statistic
a number calculated from the data that is used to evaluate compatibility of the results from null hypothesis expectation
(ie. if 14 of 18 frogs were right handed with 4 being left handed and the study was about proportion of righthand frogs than 14 is test stat)
null distribution
the sampling distribution of outcomes for a test statistic based on null hypothesis being true
what is the aim for a hypothesis test
to know how unusual our results are compared to the null hypothesis
when should we reject the null hypothesis
when the data is less than the P value (0.05)
when should we NOT reject the null hypothesis
when the data probability is LARGER than the P value (0.05)
what is P value
the probability of obtaining the data (what was seen in the test) if null hypothesis was true
what is significance level
the threshold used to determine how small P has to be to reject the null hypothesis
when is a hypothesis “statistically significant”
when the null hypothesis is rejected (less than P)
when is a hypothesis test “not statistically significant”
when the null hypothesis is NOT rejected (greater than P)
what is always included in report of hypothesis test
- value of test statistic
- sample size
- P value
if we reject a null hypothesis is it ALWAYS false
NO
if we DO NOT reject a null hypothesis is it always true
NO
what can lead to incorrect conclusions about statistical significance
sampling error (which happens because we are dealing with samples of population NOT the population itself)
what is a type 1 error
when a null hypothesis is false but taken as true (hypothesis should be rejected but isn’t)
what is a type 2 error
when a null hypothesis is true (not rejected) but taken as false (rejected)
what error is associated with false positive
type 1 (rejecting a true null hypothesis)
what type of error is associated with false negative
type 2 (when a true but gets rejected)
what is the probability of making a type 1 error
the significance level (alpha)
what happens if you try to lower the type 1 error
type 2 error rates rise
can you reduce both types of errors in an experiment
NO - decreasing one causes the other to rise
why is alpha (0.05) set so low
because type 1 errors (false positive - seeing effect when isn’t one) are seen as more serious than type 2 errors
what is statistical power
Power is the probability of CORRECTLY rejecting a false null hypothesis
do we want high or low power
HIGH - want to have high chance of correctly rejecting null hypothesis
what does a lower power mean
there is a higher probability of committing a type 2 error
how to improve statistical power
- have large sample sizes
- have a large effect size (large true differences from null expectation)
- have low variability in population
how long is the null hypothesis the default
until further data suggests otherwise
what does a small and tight CI interval mean around null hypothesis
then deviation from null hypothesis is unlikely (more likely to stat not rejected)
what are one sided tests
the alternative hypothesis has parameter value on ONE SIDE of the specified null hypothesis
when is a null hypothesis rejected in a one sided test
if the data departs from the null hypothesis IN THE DIRECTION stated by the alternative hypothesis
why are one sided tests used with caution
these tests have a P value way less than two sided tests (generally smaller sample size) so their power is lower and type 1 errors are more likely
how are null hypothesis and confidence interval similar
when the null hypothesis parameter is OUTSIDE the 95% CI, the null hypothesis is generally REJECTED (smaller than alpha)
when the null hypothesis value is INSIDE the 95% CI than the hypothesis generally is NOT REJECTED
