Hypothesis Testing, Statistical Inference and Estimation Flashcards
What does a Test of Significance determine?
It determines the probability that the observed difference between a sample statistic value and the parameter value stipulated by the Null Hypothesis (p hat - population proportion) is due only to random sampling variation.
Is the test of significance based in the sampling distribution for the statistic of interest?
Yes, it is. This is why the formula is (sample statistic - hypothesized population parameter) / Standard Deviation of the sample statistic
What is the center (expected value) of the distribution of the sample statistic (statistic of interest)?
The center is assumed to be equal (or no real difference) to the null hypothesis value for the true population parameter
How do you compute the spread of the sampling distribution of the statistic of interest?
The spread is computed differently depending on your statistic of interest. For sample proportion (sigma p hat), it is calculated as square root of Po (1-Po)/n
If we assume the null hypothesis is true (which we do when conducting all tests of significance), the shape of the sampling distribution of the statistic of interest will be approximately normal if the following conditions are met:
- n*Po is greater than or equal to 10
- n*(1-Po) is greater than or equal to 10
What is the overall output from a test of significance and how do you get it?
The p-value! After completing the test of significance formula, you get a test statistic. Test statistics always have a known probability distribution that allows you to assign a probability to the observed value of the sample statistic if the value of the population parameter stated in the Null Hypothesis is true.
What is the p-value?
It is the probability that the observed difference between the sample statistic value and the value of the population parameter stipulated by the Null Hypothesis is due only to random sampling variation, assuming the Null Hypothesis is really true.
What are One-tailed tests of significance and when are they appropriate to use over Two-tailed tests?
These tests evaluate sample data as evidence for a difference in a specific direction. These tests are appropriate only if there is a theory-based reason, well documented in the scientific literature, for expecting a specific kind of difference.
Provide additional interpretations/definitions of the p-value?
- It is a measure of the strength of the evidence against the Null Hypothesis
- It is also the probability that a conclusion to reject the Null Hypothesis value is incorrect
Is the difference between statistically significant and practically important relevant?
Yes! Even though a difference between a (observed) sample statistic and a hypothesized population parameter (the null hypothesis value) may be statistically significant, the magnitude of the difference between them must be big enough to have real-world consequences; thus, be practically important
What is Effect Size?
It is the difference between an observed sample statistic and the hypothesized population parameter. It measures the magnitude of the effect (the difference) documented by a study.
Are the p-values of One-tailed tests smaller or larger than that of Two-tailed tests? And will your overall conclusion be affected between which one you pick?
They are smaller, which also makes them more likely to support a conclusion that a significant difference exists. Also no, picking either a one-tailed or two-tailed won’t affect the nature of your final conclusion as to whether there is a significant difference. ONLY in cases where the evidence against the null hypothesis is borderline (p-value is close to 0.05)
When do you usually used Two-tailed tests of significance?
When there is insufficient basis to predict the direction of the difference prior to the study.
What means that a sample statistic is “significant”?
It means that the more likely explanation for the observed difference between the statistic value and the null hypothesis value is that the true population parameter differs from the null hypothesis value.
Why are the only two possible explanations for why the value of a sample statistic differs from the hypothesized parameter value ?
- Difference due to Random Sampling Variation only (Null Hypothesis)
- Real Difference (Alternative Hypothesis aka p-value below 0.05)
What does the Type 1 Error Rate (alpha) give you?
It gives you the probability that a conclusion to reject the null hypothesis value of the true population parameter is actually incorrect. It is quantified by the p-value (meaning it is the same value as the p-value)
When is the maximum allowable Type 1 Error rate (alpha) fixed?
When the investigator decides how small the p-value must be before he or she will conclude that there is sufficient evidence to reject the Null hypothesis value for the population parameter.
What is the Power of a test of significance (1 - beta)?
The power is the probability that a test of significance will correctly reject the Null hypothesis value for the population parameter when there actually is a specified real difference between the Null hypothesis value and the true population parameter value.
What is the specified real difference based on when discussing Power?
It is based in the investigator’s judgment about what would constitute the minimum important difference.
What does is the Type II error rate (beta)?
It is the probability that a test of significance will fail to reject the null hypothesis value when there really is a difference. In other words, the test will fail to detect a real difference.
What are the two possible reasons for why a test of significance fails to reject the null hypothesis?
- There really is no difference.
- The test of significance had insufficient power to detect it, usually due to inadequate sample size.
Absolute Effect Size vs Relative Effect Size
The first one is the actual difference between the observed sample statistic and the null hypothesis value of the population parameter; while the second one, is this difference but it is presented as a percent.
Absolute Effect Size vs Relative Effect Size
The first one is the actual difference between the observed sample statistic and the null hypothesis value of the population parameter; while the second one, is this difference but it is presented as a percent.
What two steps you need to complete in order to calculate Power? Explain them.
- Determine the MSD: The minimum significant difference refers to the rejection region, depicted by the critical value of the sample statistic which defines the range of values for the sample statistic that would provide sufficient sufficient evidence to reject the Null hypothesis. This first step only states that it is unlikely that the difference isn due only to random sampling variation.
- Determine the probability of rejecting the Null hypothesis value if a specified MID exists: the minimum important difference is defined by the investigator and it refers to a difference that is sufficiently large and interesting to prompt a change in theory or standard procedures.
