Introduction to Null Hypothesis Significance Testing; Independent-Samples T Test; One- vs. Two-Tailed Testing

Question 1

What are the extimation perspectives?

Answer 1

• In what direction is the effect
• What is best estimate for size of effect (Cohen’s d= effect size, r= strength)
• How precise is estimate of population effect (CI)

Question 2

Why is null-hypothesis significance testing (NHST) important and what are 2 disadvantages of it?

Answer 2

• Important because it is Very popular, however:
• Often not very sensible.
• Not much endorsed by APA

Question 3

Why is insufficient evidence problematic?

Answer 3

• evidence in support of hypothesis may not be enough to convince us that hypothesis is true. Chance is sufficient explanation.

Question 4

What are the 2 conclusions that can arise from null-hypothesis significance testing (NHST)?

Answer 4

• Observed effect

- Observed effect results from chance

Question 5

What does NHST often ask?

Answer 5

“Would chance on its own be likely to produce an

outcome like the one observed in sample?”

Question 6

What does the observed effect represent?

Answer 6

• proportion,
• difference between experimental and
  control group means

-is real (i.e., present in population)

Question 7

What does the Observed effect results from chance mean?

Answer 7

Observed effect results from chance not present in a population

Question 8

What does H1 represent?

Answer 8

The tested hypothesis

Question 9

What does H0 represent?

Answer 9

Questioning if the results were up to chance so H1 is counteracted

Question 10

How can we rule out chance (H0)

Answer 10

• Pretend H0 is true
• take M from Sample distribution then
• SE (SD for sample distribution) σ/ √N
• Describe graph

Question 11

What is a problem of NHST?

Answer 11

• 50% outcomes cannot rule out chance.

Question 12

How does NHST say overall?

Answer 12

How likely is observed outcome under H0?

If unlikely (and direction of outcome in expected direction), reject H0 and accept H1.

Question 13

What is a characteristic of H1 and H0?

Answer 13

The treatment population is hypothetical in character.

Question 14

What is the central outcome of the NHST?

Answer 14

the p-value: it reflects the likelihood to see our means if H0 were true

p-value = p(data | H0)

Question 15

What p is small enough to reject H0?

Answer 15

in a threshold called alpha-level (α). Typically set at (less than) 5%

Question 16

When is a result of NHST statistically significant?

Answer 16

When p-value < α

therefore reject H0 and accept H1

Question 17

When is a result of NHST non-statistically significant?

Answer 17

When p-value > α

therefore stick with H0

Question 18

What is NHST perspective?

Answer 18

Observed difference in means that could reflect an effect

∆ = …

Question 19

What is the distribution of the NHST perspective?

Answer 19

The sample distribution for ∆ (NHST perspective) is not normal distribution but t-distribution;

Question 20

Does the t-distribution sample test stay the same?

Answer 20

No, shape varies slightly with sample size.

Question 21

What results do independent-sample t tests give you?

Answer 21

Independent-samples t test gives p-value

Question 22

What does the p-value from the independent-sample t test represent?

Answer 22

difference between two sample means (∆)

Question 23

How do you calculate the observed difference in means (∆)?

Answer 23

∆ = Mexperimental – Mcontrol

Question 24

What does the t-value represent?

Answer 24

indicates how many SE ∆ (observed difference in means) is away from zero.

Question 25

What does the degrees of freedom value represents?

Answer 25

Degrees of freedom reflect sample size (N – 2)

Question 26

What does the Sig. (2-tailed) value show?

Answer 26

The p-value

Question 27

When is a test statistically significant?

Answer 27

When the p-value is smaller that 0.05

Question 28

How to report independent-sample t test results?

Answer 28

t (degrees of freedom) = t value, p = Sig. (2-tailed) value

Question 29

What does the result section need to include?

Answer 29

• IV and DV
• test statistics and p-value
• effect direction
• strength of effect, precision of estimate

Question 30

Does a p-value bigger than 0.05 mean the results are insignificant?

Answer 30

No, we are uncertain about direction of effect.

Question 31

What does the p-value tells about the magnitude of H0 if it is true?

Answer 31

The difference of the magnitude (observed difference between samples means) will occur with a probability of p x 100 or larger

eg: if p = .247 and H0 is true, magnitude difference probability = 24.7% or larger

Question 32

What is a Unidirectional hypothesis?

Answer 32

A hypothesis where the direction of the potential effect is expected

Question 33

What is Bidirectional hypothesis?

Answer 33

A hypothesis where we might not be sure about the direction of the potential effect.

(2 statements)

Question 34

What is p(data | H0)?

Answer 34

‘Probability of observed (or more extreme) outcome given that H0 is true’.