Hypothesis Testing- ERRORS Flashcards

1
Q

If p </= 0.05, would our results be likely or unlikely if the null hypothesis were true?

A

Unlikely

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

If p </= 0.05 (& our results unlikely if the null hypothesis were true), would we reject or fail to reject the null hypothesis?

A

We would reject the null hypothesis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

If p > 0.05, would our results be likely or unlikely if the null hypothesis were true?

A

Likely

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

If p > 0.05 (& our results unlikely if the null hypothesis were true), would we reject or fail to reject the null hypothesis?

A

We would fail to reject the null hypothesis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Is decision-making based on p-values fallible or infallible?

A

Fallible

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What is the consequence of decision-making based on p-values being fallible?

A

That there are 2 types of errors that we can make when deciding to reject/ fail to reject the null hypothesis.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

If the reality is that our null hypothesis (based on a population) is true, & our decision (inference based on a sample) is to fail to reject our null hypothesis, would there be an error or would we be correct?

A

We would be correct

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

If the reality is that our null hypothesis (based on a population) is false, & our decision (inference based on a sample) is to fail to reject our null hypothesis, would there be an error or would we be correct?

A

There would be an error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

If the reality is that our null hypothesis (based on a population) is true, & our decision (inference based on a sample) is to reject our null hypothesis, would there be an error or would we be correct?

A

There would be an error

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

If the reality is that our null hypothesis (based on a population) is false, & our decision (inference based on a sample) is to reject our null hypothesis, would there be an error or would we be correct?

A

We would be correct

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is 1 way in which we can express the null hypothesis?

A

As H0.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is a type I error?

A

If the reality were that our null hypothesis (based on a population) is true, & our decision (inference based on a sample) was to reject our null hypothesis (we find effects that don’t exist).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What is a type II error?

A

If the reality were that our null hypothesis (based on a population) is false, & our decision (inference based on a sample) was to fail to reject our null hypothesis (we miss effects that do exist).

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

How is it possible to never confuse type I & II errors again?

A

Simply by remembering the “Boy Who Cried Wolf” analogy (& replacing the word “wolf” with the word “effect”)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

In which order did the boy who cried wolf cause type I & type II errors?

A

He caused type I errors first, then a type II error.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What happened in the “Boy Who Cried Wolf” analogy?

A

First, everyone believed that there was a wolf when there wasn’t, and then everyone believed there was no wolf when there was.

17
Q

What is another name for a type I error?

A

A false positive

18
Q

How could the “Boy Who Cried Wolf” analogy be applied to type I errors?

A

The boy would cry wolf, causing the villagers to incorrectly reject the null hypothesis of there not being a wolf.

19
Q

How could the “Boy Who Cried Wolf” analogy be applied to type II errors?

A

There would be a wolf, however, the villagers would incorrectly accept the null hypothesis of there not being a wolf.

20
Q

In what way is guarding against errors a balancing act?

A

If we set our statistical significance threshold very low to reduce the chance of type I errors (e.g. p<= 5x10^-8), this increases our chance of making a type II error and missing real effects.

21
Q

What is the 5% significance threshold a tradeoff between in relation to errors?

A

Type I & type II errors.

22
Q

When can we reject the null hypothesis & say we have a significant effect?

A

If our probability (p-value) of the effect occurring by chance is less than 0.05.

23
Q

What is the chance of finding an effect at 0.05 if the null hypothesis is true?

A

Less than 1 in 20.

24
Q

When do we fail to reject the null hypothesis?

A

If our probability (p-value) of the effect occurring by chance is greater than 0.05.