Hypothesis test Flashcards

1
Q

Determine H & H0.

Are these outcomes equally likely?

A

H: are the counts /frq. of each category as expected?
H0: the probability of each group is exactly equal (or equal)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

Determine H & H0.

are groups different from each other?

A

H: are the means of X different in the two groups
H0: the mean of Xa equals the mean of Xb

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Determine H & H0.

did the outcome change?

A

H:did the mean of outcome X change across the two measurements?
H0: the mean of X1 equals the mean of X2

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

Determine H & H0.

are these two outcomes correlated?

A

H: are they the same?
H0: X and Y are uncorrelated (orthogonal)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

Determine H & H0.

are these variables related

A

H: is there a linear relationship between X and Y
H0: the coefficient of alpha in an OLS regression of Y on X (and Z…) is zero

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

Procedure for a statistical test

A

1-based on theory + H0 there is a stat of interest that you can calculate with sample
2-use theory to derive distribution of expected values (under H0). some assumption must be made
3- calculate the statistic actual value given sample
4-state likelihood of answer from sampling method, given that the H0 is true, if unlikely = reject H0

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What is the null hypothesis (H0)?

A

-what you are trying to disprove
- strong results, eliminate the possibility of the H0
- no difference, no change, small difference, no effect

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

reject the null if ( statistical significance)

A

p-value is small or near zero

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

type I error for binary decisions

A

rejecting the null hypothesis when it is true ( false positive)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

type ii error

A

failing to reject the null hypothesis when it is false (false negative)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

the probability of a type I error is determined by ____

A

significance level
- with a 99% significance threshold, type I error is less likely than with a 95% confidence

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

the probability of a type ii can be computed for a particular test statistic (i.e. H0), if given______

A

population distribution (parameterization, mean and SD)
sample size (N) and alpha ( chance of type I error)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

hypothesis test power

A

1 - ( the probability of a type ii error)
AKA - the probability that we correctly reject the null, when the null is false

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

frequentist statistics

A

developed before computers & calculators ( t-test, chi-square, F test)

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

bayesian statistics

A

rapidly developing framework for using prior expectation + evidence to make bets

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

measurement validity

A

how well your metric captures the underlying concept you are trying to measure

17
Q

internal validity

A

the degree to which the design of an experiment controls extraneous variables

18
Q

external validity

A

the degree to which effects found in an experiment generalize to other individuals, contexts, and outcomes
(ex: in sampling studies, are the times and places outside of the sampling frame represented accurately)

19
Q

Threats to external validity

interaction of selection and treatment

A

(unrepresentative responsiveness of treated pop.)

20
Q

Threats to external validity

interaction of setting and treatment

A

effect of the treatment may differ across geographic or institutional settings

21
Q

Threats to external validity

interaction of history and treatment

A

effect of the treatment may differ across time periods

22
Q

Threats to external validity

effect may not persist

A

-as individuals and institutions adapt over time to the treatment

23
Q

Threats to external validity

Partial- Equilibrium Effect

A

other components of the system also undergo related changes, reducing or eliminating the effect