Hypothesis Testing Flashcards
Null Hypothesis
A hypothesis that is tested in an experiment
Null Hypothesis notation
Ho: X=B
Signifigance level
Can be notated as α. 1.96 is a “critical value”
Critical Value
Test Statistic
The number that can be greater or less than a critical value, and thus be significant or not significant
Rejection Region
The value below/above the critical value. If test statistic is in this region, we can reject the Null Hypothesis. If It’s not, we fail to reject the null hypothesis. (That term is specific, we cannot “accept” a null hypothesis, only fail to reject it.”)
Types of logical “errors”
Type I: If there is not a difference, a significance, and you find that there was, you made a type one error
Type 2: If there is a difference, and you find there isn’t, you made a type two error.
α
Whatever alpha is, I’m willing to accept
beta
α level
The largest probability of committing type 1 error we “allow”, while rejecting/not accepting the No (null hypothesis)
Alternative Hypothesis (H1)
The idea that researchers are trying to prove as an “alternative” to the null: this is something akin to what we believe to actually be true, versus the null hyp.
beta (β) error
When a null, in reality, is incorrect/untrue, yet we don’t find evidence to disprove it or prove the alt hypothesis, we have made a β error.
Cohen’s d
One measure of effect size. It tells us precisely how many SD above/below the null hypothesis of our population
Cohen’s effect size conventions
An effect size measured using Cohen’s d needs context:
d
directional tests (one-tailed test)
A hypothesis test where the alt hypoth is in the form >/< the value of the null. Indicates a “specific alternative” to the null
effect size
A stat measure of the size of the “change” that a variable has had in an experiment. Measured by the differences of scores in the population between treatment and control, given that the samples used were randomly sampled.
hypothesis testing
A structured process by which scientists draw likely conclusions about a population from observations.
critical value
If you obtain sample means beyond this value on a distribution, you would reject the null hypothesis
null hypothesis (Ho)
The assumption about what is “actually” present or occurring in a population: the goal of hypothesis testing is to make a decision to reject or accept this idea. Needs to be reasonable given the data available.
obtained value
the value of the test statistic: if the sample z-statistic (or whatever statistic you’re measuring) falls outside of the α, you reject the null (it was unlikely to occur by chance)
power
the probability of rejecting a null when it is false. Put differently, it is the probability of a randomly selected sample will show the falsehood of a null when it’s actually false.
test statistic
A formula that gives us how many SD a sample is from the value of our stated Null.
one sample z test
The statistical hypothesis test used when we know a mean and SD in a single population.
p-value
the probability of obtaining a sample value given that the null is true. In behavioral science, typically, if it’s less than 5 percent, we have a decent amount of certainty that the null is untrue
effect
In experimental studies, an effect is when you are testing an experimental group, and the means you obtain have small likelihood of happening given that a basic assumption is true (null). You have a measurable and unlikely impact on your subjects.
nondirectional tests
Hypothesis tests where your goal is simply to see whether a mean in a sample is likely different (≠) or the same (=) from the mean stated in the null. (α cutoff at 5%/1.96SD is split in half, .0250 in each tail)
one-tailed test
(directional tests), tests where the alt hypothesis needs to be the null. You place the critical value in the corresponding section of the distro: if you’re looking for
Type I error
A “false positive”: when you incorrectly reject the null hypothesis. It is true, but for some reason you found likelihood that it was not. (Like finding an innocent person guilty)
type III error
When you place the rejection region in the wrong tail. (Only occurs in one tailed tests where the rejection region is placed in one or the other)
z-statistic
A z-transformation, creates a value along the standard normal distro (M=0, SD=1)
Type II error
The incorrect decision to “keep” a null that actually isn’t true. Also called a “false negative” or beta (β) error.
One sample z-test
A statistical formula used when you know the mean and variance/SD, and a single population
test statistic
a formula that allows us to know how many SDs a sample outcome is from the null (gives us a p-value)
test statistic
statistical significance
The result of a sample mean being less than 5% likely given a true null. The result shows us that something is happening in the population we’re testing.
significance testing
The process of trying to find significance in a population by experimentation.
rejection region
The area beyond a critical value in a one-tailed or two-tailed test: if a sample mean or some other sample stat we measure falls here, the null hypothesis is rejected.
significance level
α, or the percent/distance that a sample statistic/z value must be before we are “willing to accept” that the null is false. Usually a small value: you want the likelihood that something is the case given a true null to be very small.
two-tailed tests
Tests where the alternative hypothesis (H1) is specifically above OR below the null mean/parameter. You’re declaring specifically what direction you believe the experiment or population should shift relative to the basic assumption that is on trial.
Cohen’s D calculation
Cohen’s D= M-μ/σ
