Introduction to Null Hypothesis Significance Testing; Independent-Samples T Test; One- vs. Two-Tailed Testing Flashcards
What are the extimation perspectives?
- In what direction is the effect
- What is best estimate for size of effect (Cohen’s d= effect size, r= strength)
- How precise is estimate of population effect (CI)
Why is null-hypothesis significance testing (NHST) important and what are 2 disadvantages of it?
- Important because it is Very popular, however:
- Often not very sensible.
- Not much endorsed by APA
Why is insufficient evidence problematic?
- evidence in support of hypothesis may not be enough to convince us that hypothesis is true. Chance is sufficient explanation.
What are the 2 conclusions that can arise from null-hypothesis significance testing (NHST)?
- Observed effect
- Observed effect results from chance
What does NHST often ask?
“Would chance on its own be likely to produce an
outcome like the one observed in sample?”
What does the observed effect represent?
- proportion,
- difference between experimental and
control group means
-is real (i.e., present in population)
What does the Observed effect results from chance mean?
Observed effect results from chance not present in a population
What does H1 represent?
The tested hypothesis
What does H0 represent?
Questioning if the results were up to chance so H1 is counteracted
How can we rule out chance (H0)
- Pretend H0 is true
- take M from Sample distribution then
- SE (SD for sample distribution) σ/ √N
- Describe graph
What is a problem of NHST?
- 50% outcomes cannot rule out chance.
How does NHST say overall?
How likely is observed outcome under H0?
If unlikely (and direction of outcome in expected direction), reject H0 and accept H1.
What is a characteristic of H1 and H0?
The treatment population is hypothetical in character.
What is the central outcome of the NHST?
the p-value: it reflects the likelihood to see our means if H0 were true
p-value = p(data | H0)
What p is small enough to reject H0?
in a threshold called alpha-level (α). Typically set at (less than) 5%
When is a result of NHST statistically significant?
When p-value < α
therefore reject H0 and accept H1
When is a result of NHST non-statistically significant?
When p-value > α
therefore stick with H0
What is NHST perspective?
Observed difference in means that could reflect an effect
∆ = …
What is the distribution of the NHST perspective?
The sample distribution for ∆ (NHST perspective) is not normal distribution but t-distribution;
Does the t-distribution sample test stay the same?
No, shape varies slightly with sample size.
What results do independent-sample t tests give you?
Independent-samples t test gives p-value
What does the p-value from the independent-sample t test represent?
difference between two sample means (∆)
How do you calculate the observed difference in means (∆)?
∆ = Mexperimental – Mcontrol
What does the t-value represent?
indicates how many SE ∆ (observed difference in means) is away from zero.
What does the degrees of freedom value represents?
Degrees of freedom reflect sample size (N – 2)
What does the Sig. (2-tailed) value show?
The p-value
When is a test statistically significant?
When the p-value is smaller that 0.05
How to report independent-sample t test results?
t (degrees of freedom) = t value, p = Sig. (2-tailed) value
What does the result section need to include?
- IV and DV
- test statistics and p-value
- effect direction
- strength of effect, precision of estimate
Does a p-value bigger than 0.05 mean the results are insignificant?
No, we are uncertain about direction of effect.
What does the p-value tells about the magnitude of H0 if it is true?
The difference of the magnitude (observed difference between samples means) will occur with a probability of p x 100 or larger
eg: if p = .247 and H0 is true, magnitude difference probability = 24.7% or larger
What is a Unidirectional hypothesis?
A hypothesis where the direction of the potential effect is expected
What is Bidirectional hypothesis?
A hypothesis where we might not be sure about the direction of the potential effect.
(2 statements)
What is p(data | H0)?
‘Probability of observed (or more extreme) outcome given that H0 is true’.
