Chapter 8 Flashcards
Hypothesis Testing
A statistical method that uses sample data to evaluate a hypothesis about a population
Hypothesis
A specific testable prediction about the association between two variables
What is the general goal of hypothesis testing?
To rule out chance (sampling error) as a plausible explanation for the results from a research study
BUt keep in mind that it doesn’t actually do that
Hypothesis testing uses probability to support one of two possible explanations for our findings
- The observed findings are due to random change (there dows not appear to be a real effect)… called the null hypothesis
- The observed findings cannot be explained by sampling error (there does appear to be a real effect)… called the alternative hypothesis
Hypothesis Test: Step 1
State your hypothesis about the population
* The null hypothesis, H0, predicts that the indepenent variable had no effect on the dependent variable
* The alternative hypothesis, H1, predicts that the independent variable did have an effect on the dependent variable
The Hypothesis Test: Step 2
Define the probability at which we think results indicate that there must be a true effect
* The a level establishes a criterion, or “cut-off,” for deciding if the null hypothesis is correct (typically a=.05)
* Critical region consists of outcomes very unlikely to occur if the null hypothesis is true (Defined by associations that are very unlikely to obtain (typically less than 5% chance) if no effect exists)
a (alpha) level
ha
Critical region
consists of outcomes very unlikely to occur if the null hypothesis is true
-Defined by associations that are very unlikely to obtain (typically less than 5% chance) if no effect exists
They Hypothesis Test: Step 3
- Obtain and test a sample from the population
- Compute the relevant test statistic
Test Statistic
forms a ratio comparing the difference between the population mean and sample mean versus the amount of difference we would expect without any treatment effect (standard error of the M)
ex. Z-score
The Hypothesis Test: Step 4
Compare data with the hypothesis predictions
-If the test statistic results are in the critical region, we conclude the difference is significant (an effect exists) and we reject the null hypothesis
-If the test statistic is not in the critical region, conclude that the difference is not significant (any difference is just due to chance), we fail to reject the null hypothesis
Errors in Hypothesis Tests
- Hypothesis tests are not perfect. Evem when a test statistic falls in the critical region, we aren’t certain an effect exists
- Remember: we always expect some discrepancy between a M and u, there is a risk that misleading data will cause the hypothesis test to reach a wrong conclusion, two types of errors are possible
Type 1 Errors
Occur when the sample data indicate an effect when no effect actually exists; rejecting the null hypothesis when the null is true
* Caused by unusual, unrepresentative samples, falling in the critical region without any true effect
* Hypothesis tests are structured to make Type 1 errors unlikely
Type 2 Errors
Occur when the hypothesis test does not indicate an effect but in reality an effect does exist
* We fail to reject the null hypothesis even though it was actually false
* More likely with a small treatment effect or poor study design (sample size too small)
Directional test (or one-tailed test)
- Includes a directional prediction in the statement of the hypotheses and in the location of the critical region
- Ex: The original population has u=75 points on a stats test and studying is predicted to increase the scores
-HO: Test scores are not increased
-H1: Test scores are increases
-Entire critical region would be located in the positive tail bc large values for M would demonstrate that there is an increase and we would reject the null hypothesis (z=1.645+)
P-values (probability values)
- When performing hypothesis tests we will check statistical significance by seeing if our test scores (ex. z-score) indicate a p-value of less than out a level
- -Ex. we set aplha at a=.05 and check to see if p<.05
P-value definition
- The probability of obtaining an effect at least as extreme as the one in your sample data, assuming the truth of the null hypothesis
Do not tell us the probability that we’re making a Type 1 Error
Interpreting p-value correctly
Ex. Imagine we’re testing a vaccine and out hypothesis test yields p=.05
- Incorrect: If you reject the null hypothesis, there’s a 5% chance that you’re making a mistake (is actually probably around 20-50% depending on context)
- Correct: assuming the vaccine had no effect, you’d obtain the observed difference or more in 5% of stdies due to randome sampling error
What does a hypothesis test evaluate?
The statistical significance of the results from a research study
What influences hypothesis test?
The size of the treatment effect and the size of the sample… even a very small effect can be statistically significant if observed in a very large sample (would have a very small standard error)
Effect Size
-Measure of the absolute magnitude of an effect, independent of sample size
-Hypothesis tests should be accompanied by effect size
Cohen’s d
- Is a standardized effect size
- Like a z-test, Cohen’s d measures mean difference in terms of the standard deviation
M-u/o
Cohen’s d effect sizes
- d=.2-.49… small effect
- d=.5-.79… med effect
- d=.8+… large effect
Power of a hypothesis test
- The probability that the test will reject the null hypothesis when there is actually an effect
- Likelihood we can find what were looking for depends on:
-Effect size (larger effects are easier to find)
-Sample size (larger samples make it easier to find effects)
-Alpha level (larger alpha level makes it easier to find effects)
-Non-directional vs directional hypothesis (directional tests make it easier to find effects)
If other factors are held constant, then how does sample size affect the likelihood of rejecting the null hypothesis and the value for Cohen’s d?
A larger sample increases the likelihood of rejecting the null hypothesis but has no effect on the value of Cohen’s d
Under what circumstances is a very small treatment effect most likely to be statistically significant?
With a large sample and a small standard deviation
test statistic
indicates the sample data are converted into a single, spedific statistic that is used to test hypotheses
Statistical significance tells us what?
That the results are unlikely to have occurred if there is no treatment effect
