Statistical Inference Flashcards
What is a two-tailed alternative hypothesis?
simple expect a difference to exist (group A and B will differ)
What is the null hypothesis H0?
there will be no difference
What is a one-tailed alternative hypothesis?
expect a difference and state in which direction
e.g., group A will do better than group B
What is the significance level (alpha)?
criteria to decide whether to accept/reject H0
What does a significance level (alpha) of 0.05 signify?
- minimum established by scientific community
- when you find sufficient evidence to reject null you can be 95% certain that it is due to a true difference in data, not because of experimental manipulation
- accept that 5% of time results occurred by chance alone
What does an alpha of 0.01 (significance level) signify?
- stricter level significance
- when find sufficient evidence to reject null you can be 99% certain truly is difference in data because of experimental manipulation
- but accept that 1% of time results occurred by chance alone
What is the study flow?
- estimate number of subjects needed to get reliable answer
- obtain sample(s) and assign to conditions
- collect data
- calculate basic summary statistics (central tendency and dispersion)
- choose statistical test based on types of variables and types of questions being asked
- apply the STATISTICAL TEST and obtain test statistic
- COMPARE test statistic to theoretical sampling distribution derived for the particular test you are using with a particular alpha-value as your criterion
- obtain a P-VALUE = the likelihood that the result observed is due to chance if H0 is correct (alpha is the value of p at which you are willing to reject H0 even if it is correct)
- -> p < 0.05 indicates statistical significance
- decide to accept or reject H0
- derive conclusion that answers hypothesis
What are two types of Decision Errors?
- Type I (alpha)
- Type II (beta)
What is the study flow (includes early steps)?
- state H1 (alternative hypothesis)
- define population and variables
- identify outcome variables
- state H0, null hypothesis
- declare significance level (alpha)
- estimate number of subjects needed to get reliable answer
- obtain sample(s)
- collect data
- calculate summary stats (central tendency and dispersion)
- choose statistical test based on types of variables and types of questions being asked
- apply stat test and obtain test stat
- compare test stat to theoretical sampling
- obtain p-value
- decide to accept/reject H0
- derive conclusion
What is Type I (alpha) error?
reject H0 when it’s true (more serious of two errors)
say something happened when it just happened by chance
What is Type II (beta) error?
accept H0 when it’s false (less serious)
How can errors be minimized?
- by good design
- sufficient power
- but error cannot be eliminiated
What is the t-test used for?
when comparing means for two samples
What is the unpaired t-test?
- typically have control and treatment/experimental groups, each with different subjects
e. g., one group of hypertensive patients gets a new drug (treatment group) and the other gets sugar pills (control/placebo) group - has less power than paired t-tests
How can errors be minimized?
- by good design
- sufficient power
- but error cannot be eliminiated
What are some characteristics of the t-test?
- want to determine if the difference between means for each of two groups occurred because of the treatment or chance
- -> H0: no significant difference between means
- -> H1: can be either 1 (treatment group will have a higher mean score than control group) or 2-tailed (there is a significant difference between means)
- t-test calculation basically gets the difference between 2 means and divides by standard error of the difference (square root of the average standard deviation of the two groups) - this takes into account the central tendency of the 2 groups and an estimate of the average dispersion of the data
- formula yields single value called t-statistic
- compare calculated t-statistic with theoretical sampling distribution for the t-distribution (tables are found in statistics books or online) to decide if accept/reject H0
- ->need alpha value and degrees of freedom (number of subjects minus number of parameters (2))
- -> if t-stat > table reject H0; if T-stat < table accept H0
What is the trade-off between the two types of error?
as you decrease the probability of making a Type I Error you increase the probability of making a Type II Error
*as Type I is more serious most people set the Type I Error (typically at 0.05)
When you reject H0, but H0 is true, what type of error is this?
Type I
When you accept H0 but H0 is not true, what type of error is this?
Type II
What does a confidence limit attempt to do?
- capture population parameters
- range of values around mean (or other measure of central tendency) that says X% sure that it will fall int his range using confidence levels/limits
- this is used because sample statistics only estimate population statistics, can’t actually get population statistics
What do confidence limits depend on?
- standard deviation of sample data (smaller yields narrower margin error)
- sample size (larger yields narrower margin error)
- level of confidence desired (95% o5 99%)
- -> 95% tighter (narrower margin error) than 99% (to be more certain, 99% needs bigger margin of error)
- formula to calculate confidence limits depend son type of data and test
What is statistical power?
Power (1-B) = probability that you correctly reject H0 when H1 true
- typically set at 80% : power of 80% means that when H0 is truly false and there is a true treatment/experimental effect, a significant difference will be detected 80% of the time
- increase power reduces the probability of making Type II error
Increasing power reduces the probability of making which type of error?
Type II
What are methods of increasing power?
- lower alpha to 0.1: easier say difference significant - generally not accepted as 0.05 is minimum
- 1-tailed instead of 2-tailed H1 - sometimes feasible
- increase effect size (differences between means) - can’t control
- decrease variance/standard deviation - can’t control
- increase sample size = best and most effective method
- can work backwards before starting experiment and calculate what sample size would give you sufficient power
- -> need significance level (alpha)
- -> estimated std dev (from literature)
- -> estimated effect size (from literature)