Chapter 9 Flashcards
Simple vs Composite Hypothesis
- A simple hypothesis completely specifies the probability distribution
- A composite hypothesis does not fully specify the distribution;
For example, H₀: μ = μ₀ is a simple hypothesis, whereas H₀: μ > μ₀ is a composite hypothesis.
Type I & Type II errors & Powers
- Type I error: rejecting H₀ when it is true.
- Type II error: failing to reject H₀ when it is false.
- The Power of the test is thee probability of rejecting H₀ when it is false (1-β). For simple hypotheses the power is a single value, but for composite hypotheses it is a function being defined at all points in the alternative hypothesis.
* The level of significance of the test, denoted α, is the probability of committing a Type I error
* The probability of committing a Type II error is denoted β
* An ideal test would minimize both α and β, but in practice, this is not fully achievable.
In Binary Classifications
(a common situation in healthcare as well as in machine learning contexts) Here, rather than gathering a data sample consisting of multiple observations to assess whether a (population-level) hypothesis holds, a decision is required for each individual observation.
In a medical context, the classification is into healthy and diseased based on a binary test result. In these contexts:
A Type I error, known as a false positive, occurs when a healthy individual receives a positive test result; and
A Type II error, known as a false negative, occurs when a diseased individual tests negative for the disease.
Sensitivity=Power (β)=No of True Positives/No of actual positives
Specificity=1-α = No of True Negatives/No of actual negatives
Neyman-Pearson Lemma
If C is a critical region of size α and there exists a constant k such that L₀ / L₁ ≤ k inside C and L₀ / L₁ ≥ k outside C, then C is the most powerful critical region of size α for testing the simple hypothesis H₀: θ = θ₀ against H₁: θ = θ₁.
A Neyman-Pearson test rejects H₀ if:
(Likelihood under H₀) / (Likelihood under H₁) < critical value.
LIkelihood ratio test
The critical region (and test statistic) for the test is determined by setting an upper bound on the ratio (max L₀ / max L), where max L₀ is the maximum likelihood under the null hypothesis and max L is the overall maximum likelihood across all possible parameter values.
The likelihood ratio test rejects H₀ if:
(max likelihood under H₀) / (max likelihood under H₀ ∪ H₁) < critical value.
Non-parametric test
a test which doesn’t make any distributional assumptions about the data or the test statistic. These tests are called non-parametric tests.
Chi Squared goodness of fit test
Chi-squared goodness-of-fit tests are used for categorical or count data.
The test compares observed frequencies with expected frequencies under a specified hypothesis, using the test statistic:
Σ ((Oᵢ - Eᵢ)² / Eᵢ),
where Oᵢ and Eᵢ are the observed and expected frequencies for category i.
Under the null hypothesis, this statistic follows an approximate chi-squared (χ²) distribution.
The degrees of freedom (df) are calculated as:
df = (number of categories - 1) - (number of estimated parameters).
The chi-squared approximation is more accurate when expected frequencies are not too small. A common rule is that all expected frequencies should be at least 5. If necessary, categories can be combined to meet this condition.
Contingency Table
Very similar to chi squared test
A contingency table is a two-way table used to classify sample items according to two categorical variables.
The goal is to test whether the two classification criteria are independent.
Null Hypothesis: H₀: The two classification criteria are independent.
The expected frequency for any cell is calculated as:
(Row total × Column total) / Table total
If the two criteria are independent, the expected frequency for cell (i,j) is:
Eᵢⱼ = (row totalᵢ × column totalⱼ) / table total.
The degrees of freedom (df) for a table with r rows and c columns is:
df = (r - 1)(c - 1).
This method is commonly used in tests for differences among two or more population proportions.
