Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

PSYC30013 - Theme 3: P-values and NHSTs > W10 - P-Values and NHSTs > Flashcards

W10 - P-Values and NHSTs Flashcards

1
Q

In a trial-by-jury: What is the null hypothesis analogous to

A

Presumption of innocence

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

In a trial-by-jury: What is the alternative hypothesis analogous to

A

Commiting the Crime

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

In a trial-by-jury: What is the alpha value analogous to

A

Beyond a reasonable doubt

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

In a trial-by-jury: What is the trial evidence analogous to. How is it collected?

A
  • p*-value
  • Collected by data and expressed as a p - value from the relevant probability distribution under the assumption of the null hypothesis (Central distribution?)

Student’s t50 = 2.403, p = .02

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

In a trial-by-jury: What is the comparision between p-value and alpha value analogous to. Can p = .05?

A

Comparision of trial evidence and beyond a reasonable doubt criterion

  • note: obtained p will never be precisely .05 with continous distributions
  • If p < alpha, the null is rejected in favour of the alternative hypothesis. If p > alpha, the null is not rejected.
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

In a trial-by-jury: What is the decision analogous to

A

Guilty: Rejecting the Null Hypothesis

Not Guiltry: Not Rejecting the Null Hypotehsis

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

In a trial-by-jury: What is the a type 1 error analgous to

A

Type 1 error: p < alpha, but the null is true

  • The person is found guilty but did not commit the crime
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

In a trial-by-jury: What is the a type 2 error analgous to

A

Type 2 error: p > alpha, but the null is false

  • The person is not found guilty but did commit the crime
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

Do we know the Null Hypothesis is true/not in practice?

A

No.

But we act in terms of it being true/false based on evidence provided by the data.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

What are null hypothesized value

A

One value that we are proposing for the unknown population parameter

It does not have to be zero.

We use tests to examine how our sample statistic compares to this proposed population parameter

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

Define p value and its equation

A
  • p-*value is the conditional probability of the sample effect size being observed or one larger, given the null hypothesis is true.
  • p* = Pr (Tobs | H0 = True)
  • Tobs = Effect size or one larger
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is the observed test statistic value for a student’s t

A

Student’s tobs = [(M1 - M2) - (μ1 - μ2)] / SEM1-M2

  • M1 - M2
    • ​Difference in means between group 1 and group 2
  • μ1 - μ2
    • Null hypothesis size difference. While this is mostly 0, it can be non-zero.
  • SEM1-M2
    • ​Standard error of mean difference
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What does “…or one larger” suggest about the p values

A

P Values contained

  • Estimated effect size observed in the data
  • Larger-sized effects that are not observed in the data
    • Both features are required to be able to calculate the P-Value
    • P-Values contain more than what is observed in the data…
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

Define Alpha Value and its equation. In other words…

A

Alpha value is the conditional probability of rejecting the null hypothesis, given that the null hypothesis is true

a = Pr (Rejecting H0 | H0 = True)

In other words.. alpha value is the probability of type 1 error IF h0 = true

  • Both alpha and p value are conditional that H0 = true
  • But alpha is based on an action (Rejecting H0) but p-value is based on observed data (Tobs )
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What does alpha value set to achieve.

A 
# * Define the _maximum_ chance of _falsely rejecting_ a _**true** null hypothesis_ over the long run when _all statistical assumptions hold_
* Controlling how much we are wrong

Alpha is NOT type 1 error rate. It is the probability of type 1 error IF the null hypothesis is in fact true!

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

If alpha = .05, what does it mean

A

Over a large number of independent samples, the probability of false rejecting a true null hypothesis will be 5% over the long run.

i.e. commiting a type 1 error IF the null hypothesis is true

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

Define type 1 error and its equation

A

Type 1 error is the conditional probability that the null hypothesis is true given that the null hypothesis is rejected

  • Pr (H0 = True | Rejected H0)
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

How can we best distinguish Type-1 Error and Alpha Value?

A
  • Alpha Value
    • Pr (Rejecting H0 | H0 = True)
    • Set before analysis
    • Value of alpha is unchanged to what it was set to regardless of Type 1 Error
  • Type 1 Error
    • Pr (H0 = True | Rejected H0)
    • Occur after analysis and a decision is made to reject the null hypothesis.
      • If null is not rejected, probability of type 1 error = 0
      • If null is not true, probability of type 1 error = 0
19
Q

If the null hypothesis is true, what will be the distribution of p values

A

It will be uniform

(each p-value will be an equal probability of being obtained)

20
Q

If the null hypothesis is false, what is the distribution of p-values

A

It will not be uniform. Might result in high type 2 error.

21
Q

How would I increase my rejection rate?

(If I somehow knew that H0 = False)

A
  1. Increase sample size
  2. Increase diference at a population level
22
Q

What is the meaning of a p-value (in relation to population paramter)

A

Measure of consistency/compatability of sample data with the null-hypothesized population parameter.

The smaller the p, the less consistent the data with the null hypothesis…

23
Q

What does the binary decision to reject/not reject do?

A

Places a boundary on the frequency of making an incorrect rejection of a true null hypothesis over the long run when all statistical and research design assumptions have not been violated

24
Q

What are factors affecting the p-value

A
  1. Actual effect size/true state of affairs
  2. Statistical assumptions
  3. Research design assumptions (such as sample size)
25
Q

What are the misintepretations of p-values discussed in the lecture?

A
  1. Tells us null hypothesis is true or false
  2. Observed effect is due to chance alone
  3. p>alpha means no effect
26
Q

Misinterpretation of p-value 1: Tells us null hypothesis is true or false. Elaborate

A

We cannot use p to make any probabilisitc statement about the truth/falsity of the null hypothesis.

To do so, we require Pr (H0 = True | Tobs)

However,

Pr (Tobs | H0 = true) /=/ Pr (H0 = True | Tobs)

27
Q

Misintepretation of p-value 2: The observed effect is due to chance alone

A

p-value: Both assumed null hypothesis plus all statistical and research design assumptions

  • P-value is calculated on assumption that chance alone is operation
  • Does not indicate the probability of chance being the explanation (i.e. could be assumptions)
28
Q

Misintepretation of p-value 3: P > alpha = no effect is present

A

There are large number of null-hypothesized values for the population paramter than can result in P > alpha in a single sample

It does not tell us the true state of affairs.

Only circumstance which we can say there is no effect present will only occur if sample statistic is EXACTLY equal to null hypothesized values, which in practice will never occur.

29
Q

What is a p-value function plot

A

Plot of obtained p values against null hypothesized values

30
Q

If the obtained P-value = 1, what does it mean in a p-value function plot. How does the p-value change as a function of null hypothesized change

A
  • It is the peak in the p-value function plot
  • It is when the
    • Null hypothesized value = Sample Statistic
      • Comptabile (Not rejected)
    • As null hypothesized values move away, it is has lower p values
      • Imcompatible (Rejected)
    • And thus it is in that sense CI is the range of values that would NOT be rejected
31
Q

What does the bounds of the confidence interval tell us

A

It tell us the values in which p = .05

32
Q

Define confidence interval in the sense of p-values

A

Confidence intervals calculated in a single sample defines the complete set of null hypothesized values that would NOT be rejected if used in a NHST test on a sample statistic.

  • Thus, we can regard a CI as a set of plausible values for the unkown population parameter value in that they are null hypothesized values that would not have been rejected
33
Q

What are the misinterpretations of a confidence interval discussed in the lecture

A
  1. It has a 95% of capturing the true effect size
  2. Any value outside the interval are population effect that are ruled out
34
Q

Misinterpretation of confidence interval 1: It has a 95% of capturing the true effect size

A
  • Any single confidence value either (a) Capture unknown population parameter (b) Does not
    • Over the long run, 95% of all confidence intervals calculated from independently-replicated samples will contain the true effect size
35
Q

Misinterpretation of confidence interval 2: Any value outside the interval are population effects that are ruled out

A
  • Values not captured = Null-hypothesised values that would be rejected in a NHST
    • Does not mean values can DEFINITELY ruled out as population effect sizes
    • Does not mean values outside the interval have a 5% chance of being true
36
Q

What is the formula for lower and upper bound of CI. How does sample size and variability relate to it?

A

Bounds

  • θobs +- (TCrit x σSE )
    • Since σSE = (σ / sqrt n)
      • As σ increases, CI width increases, less precise
      • As n increases, CI width decreases, more precise
37
Q

Define Coverage Rate in Confidence Intervals

A

Proportion of the time that the confidence interval contains the true populaton parameter value

38
Q

If h0 = true, as n increases, what happens to rejection rate, CI coverage rate, and confidence intervals

A

Rejection rate

  • Same for each sample size

CI coverage rate

  • Same for each sample size

Precision of CI (Width)

  • Narrower Width
  • More precise confidence intervals on average
39
Q

If h0 = false, as n increases, what happens to rejection rate, CI coverage rate, and confidence intervals

A

Rejection rate

  • Increases as n increases
    • Note: Rejecting is a correct decision

CI coverage rate

  • Same for each sample size

Precision of CI (Width)

  • Narrower Width
  • More precise confidence intervals on average
40
Q

If h0 = false, as n increases AND effect is larger, what happens to rejection rate, CI coverage rate, and confidence intervals

A

Rejection rate

  • Increases as n increases
    • Note: Rejecting is a correct decision
    • Larger effect leads to larger rejection

CI coverage rate

  • Same for each sample size

Precision of CI (Width)

  • Narrower Width
  • More precise confidence intervals on average
    • No difference of effect
41
Q

If h0 = true, if n is balanced and homogenegity is violated, what happens to rejection rate, CI coverage rate, and confidence intervals

A

Rejection rate

  • Does not change with variability
  • As expected

CI coverage rate

  • Does not change with variability
  • As expected

Precision of CI (Width)

  • Greater variability = Less precise CI (Wider width)
  • Smaller variabilty = More precise CI (Narrow width)
42
Q

If h0 = true, if n is unbalanced and homogenegity is met, what happens to rejection rate, CI coverage rate, and confidence intervals

A

Rejection rate

  • Does not change with variability
  • As expected

CI coverage rate

  • Does not change with variability
  • As expected

Precision of CI (Width)

  • As expected
43
Q

If h0 = true, if n is unbalanced and homogenegity is violated, How do we know the direction of skew? How does it affect rejection and coverage rates?

A

Group 1

  • Smaller Sample
  • Smaller Variance
    • Negative Skew
      • Rejection Rate: Smaller than expected (<5%)
      • Coverage Rate: Larger than expected (>95%)

Group 2

  • Smaller Sample
  • Larger Variance
    • Positive Skew
      • Rejection Rate: Larger than expected (>5%)
      • Coverage Rate: Smaller than expected (<95%)

By defintion, since interval estimator does not equal expected rate defined by level of confidence, it is biased

44
Q

If h0 = true, if n is unbalanced and homogenegity is violated, what happens to rejection rate, CI coverage rate, and confidence intervals

A

Positvely or Negatively skewed

Rejection rate

  • Much smaller or larger

CI coverage rate

  • Much smaller or larger

Precision of CI (Width)

  • Wider or Narrow (Depends on variability and sampling size)

PSYC30013 - Theme 3: P-values and NHSTs (2 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions