Chapter 16 Significance Testing - A Real Effect?

Question 1

Alpha

Answer 1

Percentage of the probability area under H0 that forms the ‘rejection region’; level set for acceptable probability of Type 1 error under H0

Question 2

Beta

Answer 2

If the null hypothesis is not true, this is the probability that a Type 2 error will be made

Question 3

Significance Test/Decision

Answer 3

Test performed in order to decide whether the null hypothesis should be retained or rejected

Question 4

Significance

Answer 4

A significant effect is declared when the probability of the finding occurring under H0 is less than the set level of alpha

Question 5

Inferential Test/Statistics

Answer 5

Procedures for making inferences about whole populations form which samples are drawn. e.g., significance tests

Question 6

Null Hypothesis

Answer 6

Assumption of no effect in the population from which samples are drawn (e.g., no mean population difference or zero correlation)

Question 7

Alternative Hypothesis (H1)

Answer 7

Assumption than an effect exists (e.g., that populations differ or population correlation is not zero)

Question 8

Logical Probability

Answer 8

A measure of probability calculated from logical first principles

Question 9

Probability

Answer 9

A numerical measure of pure ‘chance’ (randomly based) occurrence of events

Question 10

Empirical

Answer 10

A measure of probability based on existing frequencies of occurrence of target effects

Question 11

Significance Levels

Answer 11

Levels of probability at which it is agreed to reject H0. If the probability of obtained results under H0 is less than the set level, H0 is rejected.

Question 12

Critical Value

Answer 12

Value that the result of the test statistic (e.g. z) must reach in order for the null hypothesis to be rejected

Question 13

Non-Directional Hypothesis

Answer 13

Hypothesis that does not state in which direction a difference or correlation exists

Question 14

Probability Distribution

Answer 14

A histogram of the probabilities associated with the complete range of possible events

Question 15

Rejection Region

Answer 15

Area of (sampling) distribution where, if a result falls within it, H0 is rejected; the more extreme area cut off by the critical value

Question 16

Type I Error

Answer 16

Mistake made in rejecting the null hypothesis when it is true

Question 17

Type II Error

Answer 17

Mistake made in retaining the null hypothesis when it is false

Question 18

Effect Size

Answer 18

The size of the effect being investigated (difference or correlation) as it exists in the population

Question 19

Power

Answer 19

1-Beta. The probability of not making a Type II error if a real effect exists; the probability of obtaining a case or sample statistic above the level cut off by beta in the population defined by the alternative hypothesis.

Question 20

p is less than or equal to .1

Answer 20

Significance level generally considered too high for rejection of the null hypothesis but where, if p under H0 is this low, further investigation might be merited

Question 21

p is less than or equal to .05

Answer 21

Conventional significance level

Question 22

p is less than or equal to .01

Answer 22

Significance level preferred for greater confidence than that given by the conventional one and that should be set where research is controversial or a one-shot-only trial

Question 23

Directional Hypothesis

Answer 23

Hypothesis that states which way a difference or correlation exists - e.g. population mean A > population mean B, or correlation is negative

Question 24

One-tailed test

Answer 24

Test referring to only one tail of the distribution under H0; may be used if the alternative hypothesis is directional (but controversial)

Question 25

Two-tailed test

Answer 25

Test referring to both tails of the probability distribution under H0; must be used if alternative hypothesis is non-directional