Lecture 6: p-values and confidence intervals

Question 1

What is a point estimate?

Answer 1

An inference made about the population based on the sample.

Question 2

Left-skewed distribution

Answer 2

a distribution that has a concentration of data on the upper end and the tail on the left

Skewness depends on where the tail is (so tail on left is left skewness)

Question 3

What does correlation measure?

Answer 3

Correlation measures the degree of relationship between two or more variables

It looks at association

Question 4

What is the goodness of fit of a model?

Answer 4

Goodness of fit describes how well data fits a set of observations

Question 5

What is standard error of regression?

Answer 5

The standard error of the regression (S), also known as the standard error of the estimate, represents the average distance that the observed values fall from the regression line. Conveniently, it tells you how wrong the regression model is on average using the units of the response variable

Question 6

R^2 (Goodness of Fit)

Answer 6

R^2 is a goodness-of-fit statistic.
Values: between 0 and 1.
Interpretation: The larger the better.
Meaning: Proportion of the outcome’s variability that the model explains.

Question 7

What are some questions to think about for generalising?

Answer 7

*Would I get the same coefficient if I built my model using different data?

*How likely am I to estimate the correct value?

Question 8

confidence interval

Answer 8

statistical range, with a given probability, that takes random error into account

A confidence interval refers to the probability that a population parameter will fall between a set of values for a certain proportion of times.

Question 9

What is meant by interval width?

Answer 9

Interval width = boundaries of your sample’s estimate

Question 10

When should you be concerns about confidence intervals?

Answer 10

Be concerned about confidence intervals if they include a contradictory estimate

Question 11

Match the confidence level with the interval width

99%
90%
10%

Answer 11

99% - very wide
90% - wide
10% - narrow

Question 12

Confidence level is proportional to interval width

ANALOGY

Answer 12

Analogy: A bigger net is more likely to catch the fish

you are looking for.

Question 13

If we assume there is no association, what will you expect?

Answer 13

Assuming there is no association, you will expect:

• a zero coefficient is very likely
• tiny coefficients are somewhat likely
• and big coefficients are unlikely

Question 14

What is p-value?

Answer 14

the probability of a coefficient at least as big as estimated assuming the coefficient is actually zero

Question 15

Small p-value

Answer 15

1.When we assume the True coefficient is zero, the probability of sampling 0.32 is small, i.e. our estimate is unlikely.

2.But we know our sample estimate is 0.32.

3.Therefore, we concede that our assumption is probably wrong.

4.We conclude that an association is likely

Question 16

Large p-value

Answer 16

1.When we assume the True coefficient is zero, the probability of sampling 0.32 is high, i.e. our estimate is likely.

2.And we know our estimated coefficient is 0.32.

3.Therefore, we concede that our assumption could still be right.

4.We conclude that an associations is not likely.

Question 17

Difference between small p-value and large p-value

Answer 17

Small p-value:

*Zero-assumption is probably

wrong.

*An association is likely.

Large p-value:

*Zero-assumption is probably right.

*An association is unlikely

Question 18

Example:

p-value is so small that it is <0.001

Answer 18

Typically, p-value <0.05 is statistically significant, the threshold is called alpha

GRL

Question 19

What do neither confidence intervals nor p-values tell us?

Answer 19

Neither confidence intervals nor p-values tell you anything about the effect of the exposure or how meaningful your conclusion is.

Question 20

What is the order for dealing with data?

O
G
T
A

Answer 20

Observe - look at data
Guess - build a model to estimate relationship
Test - how well does model fit (R2 and S)
Assess - confidence intervals etc

Question 21

R^2 summary

Answer 21

R^2
large = good
small = bad

Question 22

Confidence interval summary

Answer 22

If a confidence interval includes contradiction = bad

Question 23

P-value summary

Answer 23

P-value summary
Small = good
Large = bad.

Question 24

Relationship between confidence level and confidence interval width

Answer 24

The larger your sample, the more sure you can be that their answers truly reflect the population. This indicates that for a given confidence level, the larger your sample size, the smaller your confidence interval

Question 25

What does it mean if the confidence interval for a coefficient encompasses zero?

Answer 25

Zero is the null value of the parameter (in this case the difference in means). If a 95% confidence interval includes the null value, then there is no statistically meaningful or statistically significant difference between the groups.

Question 26

What does a regression coefficient mean?

Answer 26

Regression coefficients are estimates of the unknown population parameters and describe the relationship between a predictor variable and the response.

Question 27

What does it mean if the P-value is less than your significance level?

Answer 27

If the P value is less than your significance (alpha) level, the hypothesis test is statistically significant –> reject null

(GRL)