7 | Statistics for Proportions and Frequencies II Flashcards
(POLL)
How many times, on average, for 100 reports in scientific papers the confidence interval does not contain the true population value?
* never
* 1 time
* 5 times
* 10 times
* always
5 times
(POLL)
Does the 95% confidence interval allow us as well to judge on the significance level for 0.05 ?
Yes
can be used to judge statistical significance at the 0.05 significance level.
Here’s how:
Key Idea:
hypothesis test at α = 0.05 rejects null H₀ if p-value < 0.05.
95% CI provides range where true parameter is likely to lie with 95% confidence.
Decision Rule:
If the H0 value (e.g., 0 in a difference test) is outside 95% CI → Reject H₀ (p-value < 0.05, significant result).
If the H0 value is inside 95% CI → Fail to reject H₀ (p-value ≥ 0.05, not significant).
Example 1: Significant Result
Suppose a mean difference CI is (2.5, 5.8).
Since 0 is not in the interval, we reject H₀ (p < 0.05).
Example 2: Not Significant
Suppose a CI is (-1.2, 3.5).
Since 0 is inside the interval, we fail to reject H₀ (p ≥ 0.05).
Advantages of CI over p-values:
- CIs show the effect size (not just whether something is significant).
- CIs provide more information than just a yes/no decision.
(POLL)
Assuming a dependency of two variables, the same effect size and the same variance assumed, larger sample sizes gives us …
* lower p-values
* larger p-values
* the same p-value
- lower p-values
(POLL)
Can logistic regression be used instead of a Chisq-Test to evaluate the dependency between two variables?
Yes
(POLL)
To compare two distributions with values of 0 and 1 only we can use the …
* Chisq-Test
* Fisher-Test
* Kolmogorov-Smirnov-Test
* T-Test
- Kolmogorov-Smirnov-Test
(a and b could be as well seen as true, but they check if these variables are independent not the distributions)
(POLL)
To determine the effect size for a 3x2 contingency table we can use …
* Cohen’s d
* Cohen’s h
* Cohen’s w
* Cramer’s v
- Cohen’s w
- Cramer’s v
(Cohen’s h: for 2x2 table only)
(POLL)
We observe a 80 to 20 ratio in our sampling, but we expected a 50/50 ratio, what is the effect size Cohen’s w:
* 0.1
* 0.2
* 0.4
* 0.6
* 1.0
w = sqrt( ∑ [ (p_obs - p_exp)^2 / p_exp] )
= sqrt(0.36)
= .0.6
(POLL)
Simpson’s Paradoxon can happens
* during watching TV
* in un-balanced experimental design with unequally sized groups
* if a third confounder variable interfer with our two variables of investigation
* and can be usually ignored
* and might alter the interpretation of the result
* and might be removed by splitting the data based on the confounder variable
- in un-balanced experimental design with unequally sized groups
- if a third confounder variable interfer with our two variables of investigation
- and might alter the interpretation of the result
- and might be removed by splitting the data based on the confounder variable
https://www.arte.tv/de/videos/107398-002-A/mathewelten/
(POLL)
You observe zero blood pressure increases in your patients in 30 observations. What is the 95% confidence interval ou state for the confidence limits to make conclusions about the population.
* 0, 0.05
* 0, 0.1
* 0, 0.2
* 0.1, 0.5
* 0.25, 0.35
0, 0.1
3/30 = 0.1
rule of three CI95% = [0, 3/N]
_________________________
rule of three:
* if the numerator is zero (zero observations) rule of
three: CI95%[0,3/n]
* if the numerator is one (one observation) rule of
five CI95%[0,5/n]
* if the numerator is two (two observations) rules of
seven CI95%[0,7/n
Confidence interval?
- confidence in statistics: another way to describe probability
- mean of estimate plus & minus variation in estimate.
- range of values estimate expected in if testing again, within certain level of confidence.
What can we use to express uncertainty about our inferences? Which is more informative?
- p-values
- confidence intervals (a bit more informative - or did he mean both together?
What’s going on here?
> prop.test(6,44) # sample in 2009 1‐sample proportions test with continuity correction data: 6 out of 44, null probability 0.5 X‐squared = 21.841, df = 1, p‐value = 2.962e‐06 alternative hypothesis: true p is not equal to 0.5 95 percent confidence interval: 0.05667816 0.28046238 sample estimates: p 0.1363636
Summary of prop.test(6,44) Output
* Test: 1-sample proportion test (Chi-square with continuity correction).
* Null Hypothesis (H₀): True proportion p = 0.5 (default)
* Observed Proportion: 6/44 ≈ 0.136 (13.6%).
* Test Statistic: X² = 21.841 (df = 1).
* p-value: 2.96e-06 (very small, reject H₀).
* 95% CI: [0.057, 0.280] (true proportion likely between 5.7% and 28.0%).
* Conclusion: Strong evidence that p ≠ 0.5; observed proportion is significantly lower.
Null Hypothesis: H0 with one group?
- sample proportion not different from known population proportion
- (sample mean not different from known population mean)
alternative hypothesis: - same sentences but without not
Null Hypothesis: H0 with two or more groups?
- groups not different from each other regarding population proportions (or means)
alternative hypothesis: - same sentences but without not
p-value according to D
gives probability to observe a difference as large or even larger as seen with the sample if H0 would be true
Can we reject the null hypothesis?
Based on our sample data we will never know if the null hypothesis is true or false‼
We almost always observe a difference‼
What does the p-value tell us when testing one group?
We need to know the population mean/proportion!:
Tells us how sure we are that our sample is not different from the population.
Can we reject the null hypothesis?
Based on our sample data we will never know if the null hypothesis is true or false‼
We almost always observe a difference‼
What does the p-value tell us when testing two groups?
how often get observed or larger difference by random sampling from two populations if means/proportions of both populations would be equal.
What does the p-value not tell us?
how sure we are that there is really a difference!
Common threshold for significance?
- statistically significant does not necessary tell us that the observation is “important”
- significance level α: just custom threshold for p-value to claim significance (mostly <0.05)
- α states the probability of falsely rejecting the null hypothesis
p-value of 0.04 → random sampling of two _________ populations would observe in __ of all sampling procedures an _____________ → significant
p-value of 0.04 → random sampling of two identical populations would observe in 4% of all sampling procedures an equal or larger difference → significant
even if something is not significant don’t stop thinking‼ What must one be aware of?
- p-value of 0.06 → not significant
- significant * (<0.05), highly significant ** (<0.01). extremely significant *** (<0.001)??
- it might be that a fivefold change can be just significant, whereas a 1.05 fold change can be extremely significant ….
- with same fold changes but larger sample size you can go from significant to highly significant
- doesn’t tell you anything about the strength of the effect !
Rosnow and Rosenthal (1989): “Surely God loves the .06 nearly as much as the .05!”
Significance, Importance and Scatter:
Large _______ can result in ________ even with larger changes, possibly more important results!
Large scatter can result in ‘non-significant’ even with larger changes, possibly more important results!
How does sample size affect significance? Here is a hint:
> set.seed(1234) > s1=rnorm(1000,mean=10) > s2=rnorm(1000,mean=11) > t.test(s1,s2)$p.value [1] 2.826639e‐108 > s1=rnorm(100,mean=10) > s2=rnorm(100,mean=11) > t.test(s1,s2)$p.value [1] 7.321056e‐09 > s1=rnorm(20,mean=10) > s2=rnorm(20,mean=11) > t.test(s1,s2)$p.value [1] 0.05413113
A larger sample size will increase the significance for data sampled from the same distribution. The code shows sampling from a normal distribution.
* Sample size 1000: extremely significant
* Sample size 100: very significant
* Sample size 20: barely significant
R:
CI and p-value for one group with known population proportion. What’s going on here?
> prop.test(9,60,p=0.298) # http://de.statista.org/ 1‐sample proportions test with continuity correction data: 9 out of 60, null probability 0.298 X‐squared = 5.5948, df = 1, p‐value = 0.01801 alternative hypothesis: true p is not equal to 0.298 95 percent confidence interval: 0.07501469 0.27076912 sample estimates: p 0.15 > prop.test(9,60,p=0.298)$p.value [1] 0.01801402 > prop.test(9,60,p=0.298)$conf.int [1] 0.07501469 0.27076912 attr(,"conf.level") [1] 0.95 > prop.test(11,60,p=0.298)$p.value # two smokers more [1] 0.07173252 > prop.test(11,60,p=0.298)$conf.int # ci does cross pop.prop [1] 0.09932066 0.30853889 attr(,"conf.level") [1] 0.95
Smoker students!
1st test: prop.test(9,60, p=0.298)
* example of one-sample proportion test (prop.test()),
* compares observed proportion against hypothesized population proportion.
* H0: true proportion is p=0.298
* H1 true proportion is not 0.298 (two-sided test).
* Chi-squared statistic: 5.5948
* p-value: 0.01801 → less than 0.05 → reject H0, suggesting that observed proportion significantly different from 0.298.
* 95% CI: [0.075,0.271] → 0.298 not in interval, test confirms a significant difference.
2nd Test: prop.test(11, 60, p = 0.298)
* two more individuals are counted in the sample, increasing the proportion… etc
R:
What is a prop.test?
- function performs a test of proportions, either a one-sample or two-sample test
- uses Chi-squared test
1 sample: - prop.test(x,y) → 1-sample prop test, checks if obs. prop differs from exp. prop (default = 0.5).
- x → number of observed “successes” (e.g., cases, events, etc.)
- 44 → total number of trials (or observations).
2 sample: - prop.test(c(x,y), c(a,b))