Chapter 16 Significance Testing - A Real Effect? Flashcards
Alpha
Percentage of the probability area under H0 that forms the ‘rejection region’; level set for acceptable probability of Type 1 error under H0
Beta
If the null hypothesis is not true, this is the probability that a Type 2 error will be made
Significance Test/Decision
Test performed in order to decide whether the null hypothesis should be retained or rejected
Significance
A significant effect is declared when the probability of the finding occurring under H0 is less than the set level of alpha
Inferential Test/Statistics
Procedures for making inferences about whole populations form which samples are drawn. e.g., significance tests
Null Hypothesis
Assumption of no effect in the population from which samples are drawn (e.g., no mean population difference or zero correlation)
Alternative Hypothesis (H1)
Assumption than an effect exists (e.g., that populations differ or population correlation is not zero)
Logical Probability
A measure of probability calculated from logical first principles
Probability
A numerical measure of pure ‘chance’ (randomly based) occurrence of events
Empirical
A measure of probability based on existing frequencies of occurrence of target effects
Significance Levels
Levels of probability at which it is agreed to reject H0. If the probability of obtained results under H0 is less than the set level, H0 is rejected.
Critical Value
Value that the result of the test statistic (e.g. z) must reach in order for the null hypothesis to be rejected
Non-Directional Hypothesis
Hypothesis that does not state in which direction a difference or correlation exists
Probability Distribution
A histogram of the probabilities associated with the complete range of possible events
Rejection Region
Area of (sampling) distribution where, if a result falls within it, H0 is rejected; the more extreme area cut off by the critical value
Type I Error
Mistake made in rejecting the null hypothesis when it is true
Type II Error
Mistake made in retaining the null hypothesis when it is false
Effect Size
The size of the effect being investigated (difference or correlation) as it exists in the population
Power
1-Beta. The probability of not making a Type II error if a real effect exists; the probability of obtaining a case or sample statistic above the level cut off by beta in the population defined by the alternative hypothesis.
p is less than or equal to .1
Significance level generally considered too high for rejection of the null hypothesis but where, if p under H0 is this low, further investigation might be merited
p is less than or equal to .05
Conventional significance level
p is less than or equal to .01
Significance level preferred for greater confidence than that given by the conventional one and that should be set where research is controversial or a one-shot-only trial
Directional Hypothesis
Hypothesis that states which way a difference or correlation exists - e.g. population mean A > population mean B, or correlation is negative
One-tailed test
Test referring to only one tail of the distribution under H0; may be used if the alternative hypothesis is directional (but controversial)
Two-tailed test
Test referring to both tails of the probability distribution under H0; must be used if alternative hypothesis is non-directional
